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PREFACE 


This volume is addressed to field workers; teachers, princi. 
pals, supervisors, and superintendents; graduate students who 
are preparing to be field workers; and those who supervise 
the training of such workers. The point of view of the authors 
is that competent educational workers should participate in 
the solution of professional problems, either as consumers or 
producers of research. In the rôle of consumers, emphasis is 
placed on the application of investigational results to practical 
field situations and problems. Although some field workers 
will make significant contributions to the store of educational 
knowledge as active participants in the production of research, 
the primary outcomes for the majority of field participants 
in educational research will be found in the training value of 
the problem-solving approach, with an increased understanding 
of the educative process. 

To argue that most research should be conducted in grad- 
uate schools, laboratories, and research bureaus is to under- 
estimate the importance of field situations in terms of the 
application of the results of educational investigation and the 
intellectual and professional growth of approximately one mil- 
lion teachers in this country, through the discovery and solu- 
tion of important educational problems under normal field 
conditions. From this point of view, competent field workers 
have an important dual obligation: one relating to the prac- 
tice of teaching, supervision, or administration, and the other 
relating to problem solving in their respective areas of respon- 
sibility. Then, too, if children are to learn through problem- 
solving methods and activities, the teachers themselves should 
have at least a working knowledge of the scientific method, 


such as is presented in this volume. Both intelligent consump- 
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tion of investigational literature and effective participation in 
the solution of practical problems of teaching, administration, 
and supervision require an understanding of at least the ele- 
mentary principles of problem solving and research. 

It has not been possible to supply within a one-volume treat- 
ment of the methodology of educational research all of the 
information necessary for the solution of many of the more 
technical problems of education. For these, the producer of 
research may need additional advanced training in keeping 
with the nature of the study undertaken, for example, the sta- 
tistical analysis of data, construction and use of measuring 
instruments for the collection of data, and the invention and 
manipulation of laboratory equipment. 

The preceding paragraphs should not be interpreted to mean 
that all field workers are to do research, without regard to their 
training and qualifications. In making these statements it is 
conceded that scientific experimentation is difficult and com- 
plicated, much more so than the average lay or even profes- 
sional worker realizes. It is recognized that not all teachers 
will rise to the level of making scientific contributions to the 
study of education. This statement, of course, is also true of 
many so-called research workers. There is really a tendency on 
the part of the educational practitioner to put entirely too 
much faith in the training researches of both teachers and re- 
search workers, Unfortunately, supervisors and professors to- 
day seem to assume too frequently that they alone are to see 
problems, solve problems, and do the educational thinking pre- 
requisite to the successful management of school systems. 
They hand down their solutions in the form of methods to be 
applied, textbooks to be used, courses of study to be followed, 
and policies for teachers and students to put into operation. 
The teachers, then, only teach, acting neither as producers nor 
consumers of research. If one were to visit a classroom and 
find the teacher doing all of the thinking, entertaining all of 
the points of view, and solving all the problems, he would char- 
‘acterize such a worker as inefficient. Judged from this point of 
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view, much of the supervision, administration, and instruction 
of to-day is ineffective.+ 

The authors have sought to avoid such a paternalistic atti- 
tude in the preparation of this book. An attempt has been 
made to present the major principles of investigational proce- 
dure, together with numerous illustrations and bibliographical 
references, rather than to formulate research recipes to be fol- 
lowed in routine fashion. Only in this type of treatment, which 
leaves much to the interest and initiative of the reader, could 
the authors be consistent with the true spirit of scientific 
method. i 

Convinced of the feasibility of improving educational pro- 
cedure through problem solving, many teachers, supervisors, 
administrators, and other field workers have lacked the neces- 
sary technical knowledge. It is the purpose of the present 
volume to supply this information so far as possible within 
the scope of an elementary treatise. It is recognized that a 
number of helpful discussions of the methodology of educa- 
tional research have appeared within the past eight or ten 
years. However, the more general books on investigational 
procedures have been rather brief and incomplete, while the 
specialized volumes devoted entirely to a single area, such as 
tests, statistical methods, the school survey, the questionnaire, 
historical research, experimental research, the interview, or 
case work, have covered only a limited portion of the field of 
research methodology. Also, a large amount of significant 
literature on research procedures has appeared within recent 
years, scattered through periodicals, bulletins, monographs, 
year-books, and theses, which the authors have attempted to 
assemble and interpret for the benefit of field workers, most 
of whom do not have access to such materials in the original 
sources of publication. 

This book presents a reasonably complete analysis of the 
literature dealing with the methodology of educational re- 


1A. S. Barr, “Research for Teachers,” Journal of Educational Research, XX 
(June, 1929), 42-3. 
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search; moreover, the authors have sought to inculcate in the 
research student a type of intellectual independence which 
comes in large part through the development of desirable 
bibliographical procedures and habits, as keys to the vast 
stores of educational literature. Therefore, constant reference 
has been made to library guides such as the Education Index, 
the Review of Educational Research, and the annual bibliog- 
raphies of research published by the United States Office of 
Education, in order that months or years after this book has 
appeared in print, the reader may bring its contents up to date 
by a canvass of current educational literature. 

The sequence of chapters here represented may be defended 
in terms of logical organization, as well as in terms of an 
analysis of the various possible ways of attacking educational 
problems, and of the steps in scientific method or problem 
solving. The introductory chapter presents a characterization 
of scientific method and its contribution to education. The 
next three chapters deal with the early stages of research; 
namely, the selection and definition of the problem, the survey 
of related information and investigations, and formulation and 
testing of hypotheses. Chapter V attempts a systematic clas- 
sification of available investigational procedures, as well as 
lists many of the classifications adopted by other writers in 
this field. Chapters VI, VII, VIII, IX, and X discuss in detail 
the major approaches to problem solving in education; namely, 
through historical, survey, experimental, and related methods 
of research. Chapters VII and VIII include descriptions of the 
various instruments for the collection of data, used quite ex- 
tensively in normative-survey investigations, and in varying 
degrees in the other types of research. Chapters XI, XII, XIII, 
and XIV deal with later steps in the research process; namely, 
the analysis and interpretation of data, formulation of conclu- 
sions and generalizations, writing of the research report, and 
evaluation of educational writing and investigation. Chapter XV 
is of concern to apprentices in educational research and to 
those who supervise their training, whether in the field or in 
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the graduate school. It is suggested that some readers may 
wish to begin with the last chapter (XVI) as an overview of 
the entire volume. 

The authors have deliberately avoided use of the term scien- 
tific on the title page and at many other points where it might 
have been employed, maintaining that there is no essential 
conflict between the methods of science and the processes of 
logic and philosophy. In clarifying the complementary func- 
tions of science and philosophy, it has seemed more effective 
to discuss the contribution of philosophy and logic in connec- 
tion with the various steps in problem solving (formulation of 
hypotheses, analysis and interpretation of data, formulation of 
conclusions and generalizations, and evaluation) rather than 
to devote a separate chapter to philosophical procedures. The 
plan of organization followed, therefore, presents philosophy in 
its functional relationship to a number of steps in problem 
solving rather than as a separate or self-sufficient method of 
research, 

Tn order to show more clearly that the various chapters rep- 
resent, in the main, different aspects of the research pattern or 
different steps in problem solving, certain related problems and 
exercises have been carried through a number of chapters. 
For example, the problem of how the school may adapt its 
program of instruction to varying needs is treated in terms of 
the formulation of a problem, the survey of related literature, 
the application of survey, experimental, and historical tech- 
niques, and of other procedures. 

It is hoped that this volume may be of service in one or 
more of a variety of educational situations: (1) possibly in 
the last year of the undergraduate teacher-training institution, 
as a preview of graduate work in education or in courses deal- 
ing with scientific method; (2) in the first year of graduate 
study, with an orientation function at an advanced level, 
somewhat like that of certain introductory courses and texts 
at the undergraduate level (for example, C. H. Judd’s Intro- 
duction to the Scientific Study of Education) ; (3) in classes or 
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seminars which deal with principles of thesis writing and 
methods of educational research; (4) in institutional or field 
groups made up of so-called “consumers” of research, since the 
fundamental principles of investigational procedure should be 
understood by both producers and consumers of research, al- 
though, of course, the former will need a more technical and 
intensive type of training in the methodology of problem 
solying; and (5) as a handbook and reference manual for 
educational workers in whatever problem-solving capacity they 
may serve. The patterns of research methodology discussed 
have been drawn from the fields of education, psychology, so- 
ciology, philosophy, logic, history, statistics, and other sciences, 
and organized as an integrated series of functional approaches 
to problem solving in education. 

Acknowledgment is made in footnotes and bibliographies to 
the large number of authors and publishers who have made 
possible at least a foundation for the ultimate establishment 
of education on a sound basis, and who have generously per- 
mitted the use of certain of their materials in the present 
volume. 

The authors are much indebted to the large number of edu- 
cational workers who have either criticized or tested out in 
their classes the experimental edition of this book: R. L, 
Johns, Alabama Polytechnic Institute; J. R. Gerberich, Uni- 
versity of Arkansas; G. M. Wilson, Boston University; T. E. 
Newland, Bucknell University; O. E. Hertzberg, State Teach- 
ers College at Buffalo; W. A. Smith, University of California 
at Los Angeles; Noel Keys and L. A. Williams, University 
of California; T. G. Foran, Catholic University of America; 
C. H. Judd, G. T. Buswell, Newton Edwards, and W. C. 
Reavis, University of Chicago; L. A. Pechstein, Gordon Hen- 
drickson, James Vaughn, G. A. Hedger, and S. L. Eby, Uni- 
versity of Cincinnati; H. P. Allen, Claremont College; R. A. 
Davis, University of Colorado; Maude Williamson, Colorado 
State College of Agriculture and Mechanic Arts; Carter Alex- 
ander and P. M. Symonds, Teachers College, Columbia Uni- 
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versity; W. A. Brownell, Duke University; W. C. Ruediger, 
George Washington University; P. J. Rulon, Harvard Univer- 
sity; Florence E. Bamberger, Johns Hopkins University; C. 
H. Thompson, Howard University; E. L. Abell, Indiana State 
Teachers College; P. V. Sangren, Western State Teachers 
College, Michigan; R. A. Schwegler, University of Kansas; W. 
S. Taylor, University of Kentucky; I. P. Foote, Louisiana 
State University; W. S. Small, University of Maryland; W. 
R. Good and H. C. Koch, University of Michigan; E. L. 
Austin, Michigan State College; A. C. Eurich and P. O. John- 
son, University of Minnesota; P. V. West, New York Univer- 
sity; H. H. Abelson, College of the City of New York; P. A. 
Witty, Northwestern University; L. F. Kuntz, University of 
Notre Dame; H. G. Good, Ohio State University; C. O. 
Mathews, Ohio Wesleyan University; C. W. Knudsen, George 
Peabody College for Teachers; R. D. Matthews and E. D. 
Grizzell, University of Pennsylvania; G. W. Hartmann and 
R. P. Wray, Pennsylvania State College; O. K. Buros, Rut- 
gers University; F. M. Crowley, St. Louis University; W. H. 
Burton, University of Southern California; J. B. Sears, Stan- 
ford University; H. P. Smith, Syracuse University; G. E. 
Walk, Temple University; J. E. Avent, University of Ten- 
nessee; B. F, Pittenger, University of Texas; E. A. Jacobsen, 
Utah State Agricultural College; E. K. Hillbrand, University 
of Wichita; M. J. Neuberg, Wittenberg College; and O. C. 
Schwiering, University of Wyoming. M. D. Munn, graduate 
assistant at the University of Cincinnati, was of material as- 
sistance in certain of the bibliographical work. 
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EDITOR’S INTRODUCTION 


The past quarter century has seen a very great increase in 
the use of scientific methods in the attack upon educational 
problems. The results of scientific research have increasingly 
affected educational procedure in all phases—administration, 
supervision, and teaching. 

The realization has grown that all school workers must know 
something of the nature and technique of scientific research. 
This is necessary, first, in order that such workers may utilize 
sound methods in attacking their own practical problems. All 
too often these problems have been solved by reference to 
“experience,” which is perforce limited and in many cases 
misleading. Critically analyzed experience is, of course, valu- 
able. Second, this knowledge is necessary in order that school 
workers may be better able to distinguish the fundamental 
from the superficial, the basic from the momentary fad. Pos- 
sessed of relatively limited and superficial training, too many 
school workers in the United States are prone to follow the 
“latest idea,” whether it be sound or hopelessly superficial. 

More recently there has emerged the idea that classroom 
teachers must not only make use of scientific methods and 
know the technical background of teaching, but should also 
be encouraged to participate in the derivation of this very 
scientific background itself. This idea is sound psychologically 
and fully in accord with our democratic school philosophy. 
Hence, knowledge of scientific methods is necessary not only 
for the field worker on any level, but also for all prospective 
school workers in training. $ 

A number of books and manuals have appeared in answer 
to the need just indicated. Most of them have been commend. 


able, pioneer efforts. The authors of the present volume have 
xiii 
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built upon and acknowledged their indebtedness to the earlier 
works, but have gone far beyond previous efforts. The material 
has been in formulation for several years and was tested in 
many institutions in mimeographed form. Each of the three 
authors has done noteworthy work in his particular aspect of 
the total field so that the volume here presented is a sound, 
systematic, coherently organized treatment of the techniques 
of scientific research. A number of troublesome items are 
treated here for the first time. The language is such that any 
school worker or graduate student can understand it. Graduate 
students in particular will be greatly aided through the sugges- 
tions for finding problems and starting research. 

The authors have confined their treatment to methods of 
scientific research even though they have deliberately avoided 
the use of the word scientific in the title. The philosophic 
method is mentioned as both supplementary and complemen- 
tary to the scientific. No discussion is given of the separate or 
complementary use of the philosophic method though the 
supplementary use is discussed at various points. The authors 
prefer to treat the two methods as parts of a complete act of 
thought and sincerely believe that nothing is gained by sepa- 
tate discussion. Although this treatment is in accord with the 
definition and practice of many scientists in education, there 
will be readers from both the scientific and the philosophic 
schools who will regret omission of separate discussion of the 
two methods. 

The volume is, as indicated above, a masterful treatment 
and should have marked effect upon those phases of school 
work susceptible to scientific attack. 

W. H. BURTON 
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CHAPTER I 
THE NATURE OF SCIENTIFIC THINKING 


Man in his search for truth has appealed, in general, to five 
sources of evidence: (x) custom and tradition, (2) authority, 
(3) personal experience, (4) syllogistic reasoning from appar- 
ently self-evident propositions, and (5) scientific inquiry.* 
These five sources of evidence represent a rough and over- 
lapping chronological sequence in the history of human think- 
ing, man’s earliest search for. truth being characterized by an 
appeal to custom, tradition, authority, and personal experience, 
and later by an appeal to syllogistic reasoning and scientific 
inquiry. Although it is evident that man has not yet devised 
any perfect methods of searching for what he calls truth, it 
appears that, with the passage of time, improvements have 
been made in this respect. In the appeal to custom, tradition, 
authority, and personal experience man found a source of 
truth not without value. As time passed, however, he dis- 
covered other sources which seemed to offer new knowledge, 
understanding, and insight. Thus arose the appeal to syllogis- 
tic reasoning and scientific inquiry. These methods of seeking 
truth have continued to the present time and constitute man’s 
current intellectual equipment for solving the mysteries of 
life and nature. 

The appeal to custom and tradition. Of the totality of 
man’s activities, comparatively few will be brought con- 
sciously to his attention in such a manner that he will need 
to choose between them, to determine their efficacy, or to seek 


1John C. Almack, Research and Thesis Writing, pp. 57-63. Boston: Houghton Mifflin 


Co., 1930. 
William A. McCall, How to Experiment in Education, pp. 1-3. New York: The Mac- 
millan Co., 1923. 3 ‘ 
William Clark Trow, Scientific Method in Education, pp. 9-27. Boston: Houghton 
Mifflin Co., 1925. 
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their ultimate logical justification. The majority of his opin- 
ions, attitudes, and actions will be as they are because he 
knows, desires, or has time to discover nothing better. The 
customs of his times and the traditions of his people will 
decide such things for him. His clothes, speech, food, and 
mode of living are largely determined in the same manner.’ 
This. acceptance of custom and tradition is to some extent a 
necessary human economy, but when the fact that a thing is, or 
has always been so, is employed as its justification, the result 
may be an appalling intellectual stagnation. When man recog- 
nized his own mental inadequacies in this respect, a great step 
forward was taken in the development of human thinking. 
The appeal to authority. In times of storm, pestilence, and 
conflict man often finds the dictates of custom and tradition 
inadequate; he is driven from his complacency to newer and 
better modes of behavior. Savage man with his belief in magic, 
charms, and the supernatural appealed to the tribal wise men 
and to his gods in times of crises. These practices have been 
succeeded in civilized nations by secular and religious au- 
thority. In early times in nearly all countries the aim, content, 
and methodology of education, for example, were determined 
by the authority of the church. In China the formal educa- 
tion of children began and often ended with a study of the 
writings of Confucius; in India, with a study of the Koran; in 
Palestine, with a study of the Old Testament; and in western 
Europe and America, with the study of the Bible.* For cen- 
turies the church which had civilized and christianized the 
western world was looked upon as arbiter in all matters whether 
religious or secular. It is natural that her pupils would turn to 
their teacher when doubts arose and accept her authority as 
final, not to be questioned by the layman.* With the growth of 
strong secular states in western Europe, however, the balance 
of power passed from church to state, and the appeal was 
2W. C. Trow, op. cit, p. 9. 


3 Ibid., pp. 9-27. 
4 Samuel Chester Parker, A Textbook in the History of Modern Elementary Educa- 


tion, pp- 109-34. Boston: Ginn and Co., 1912. 
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addressed to kings, parliaments, and legislatures. In recent 
years, for example, at least three state legislatures in the United 
States have enacted laws on the validity of the principles of 
evolution. Other sources of authority are found in the nations’ 
scholars, philosophers, and scientists, the worth-whileness of 
their counsel depending of course upon their judgment, in- 
telligence, and scholarship. From the dawn of civilization man 
has appealed (and in the absence of more reliable informa- 
tion, rightfully) to his oracles, his intellectual superiors, his 
church, his state, his king, and to his God or gods for guidance. 

The appeal to personal experience. Although man does ap- 
peal to custom, tradition, and authority in his attempts to 
derive satisfactory solutions to perplexing problems, no single 
source of evidence is, however, more universally employed in 
the search for truth than is personal experience. This source 
of evidence provides knowledge of most ordinary affairs in 
everyday life. When confronted with a difficult situation, one 
naturally turns first to his own experiences in similar situa- 
tions for an idea of what to do. If one’s own experiences are 
inadequate, he may turn to those of friends and acquaintances. 
If the problem is of sufficient importance and of long standing, 
and if there are records of the experiences of others with it, 
one may profit by an appeal to history. History offers, thus, the 
means by which the experiences of people are extended from 
the present into remote times and places. 

Valuable as the appeal to personal experience is, it is sub- 
ject, however, to gross inaccuracies. Important sources of error 
are: (1) argument from a single or limited number of in- 
stances; (2) argument from positive instances, and the neglect 
of negative instances; ( 3) the omission of evidence contrary 
to one’s opinion; (4) failure to observe important circum- 
stances attending different phenomena; ( 5) erroneous conclu- 
sions due to prepossessions, preconceived ideas, and prejudices ; 
(6) attributing to a single antecedent consequences arising 
from two or more antecedents; (7) inaccurate instruments of 
measurement, dependence upon subjective judgment, estimates, 
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and guesses; (8) argument from analogy; (9) failure to dis- 
criminate between material and immaterial circumstances; 
(10) mistaken inferences of various sorts; and (11) hasty 
generalizations from insufficient evidence. For a further discus- 
sion of these ordinary fallacies of thinking, the reader is re- 
ferred to almost any elementary textbook in logic." 

The aim of the preceding statement is not to minimize the 
importance of personal experience as a source of evidence in 
everyday affairs. Science itself, as is generally known, is an 
appeal to experience. James Harvey Robinson has already 
referred to the fact that most people do not reason, but ration- 
alize. That is, they employ their own individual experiences 
to support previously acquired likes and dislikes. This is true, 
but it is also true that a keen mind, with some knowledge of 
the fallacies of ordinary thinking, may offer many valuable 
observations out of the richness of its own experiences and 
those of others. These observations constitute not only a source 
of practical guidance, but a valuable source of hypotheses for 
scientific inquiry. This intimate connection between personal 
experience and science is frequently overlooked. In many of 
its undertakings, science merely begins with the distinctions, 
tentative generalizations, and the suppositions found in ordi- 
nary experience and extends these by making them more defi- 
nite and consistent. There is already in the language of 
ordinary life an organization of experience, and it is from this 
pattern that scientific thinking takes its departure. From this 
point of view, science consists merely of correcting, extending, 
and verifying familiar concepts of life. 

The appeal to syllogistic reasoning from apparently self- 
evident propositions.’ A great advance was made when man 
began to think about his own thinking. The product of these 
early intellectual efforts was deductive logic, founded by Aris- 


5 Thomas Fowler, The Elements of Inductive Logic. (Third edition, corrected and 
revised.) Oxford: Clarendon Press, 1876. Pp. xxvii + 360. 

6 James Harvey Robinson, The Mind in the Making: The Relation of Intelligence to 
Social Reform. New York: Harper and Bros., 1912. 

7J. Welton, The Logical Bases of Education, pp. 105-6. London: Macmillan and 


Co., Ltd., 1919. 
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totle and perfected by the medieval scholastics. The chief 
instrument of deduction was the syllogism, composed of a 
major premise, a minor premise, and a conclusion, as, for ex- 
ample, (1) all sinners deserve punishment ; (2) A B is a sinner; 
(3) therefore, A B deserves punishment. The essence of such 
syllogistic reasoning consists of showing that a given particular 
case falls under a general rule. As applied by the medieval 
scholastics, it consisted chiefly of proving or disproving, by 
the use of appropriate logical techniques, selected statements 
from Aristotle and the Church Fathers.’ Although the scholastic 
method purported to furnish good mental training, it tended to 
degenerate into mere argumentation for its own sake, and con- 
sidering the amount of attention devoted to it, contributed 
relatively little as a means of discovering truth. Defended and 
opposed by long strings of syllogisms, such hair-splitting dis- 
tinctions as whether a pig driven to market is held by the man 
or the rope, or whether a shield white on one side and black 
on the other may be called black or white, brought this system 
of logic into disrepute.” The primary weakness of the syllogism, 
however, is in its premises. These were accepted by medieval 
scholastics as authoritative or as apparently self-evident from 
experience. Granting the premises, mistakes were not usually 
made in the actual chain of reasoning. A secondary weakness 
of syllogistic reasoning arises from the indefinite, ambiguous, 
and elusive verbal symbolism of logical terms and propositions. 
Words have many interpretations, meaning different things to 
different people at different times. These and other factors 
have caused the syllogistic method to lose favor. 

Although this approach to problem solving is not generally 
in good repute to-day, the fact remains that it has done 
splendid service in the past and contains much of value for 
present and future problem solving. The chief difficulties with 
it, already noted, are its acceptance of premises as self-evident 

8 Roscoe Lewis Ashley, Early European Civilization, pp. 516-9. New York: The Mac- 
millan Co., 1917. 


9 Daniel S. Robinson, The Principles of Reasoning: An Introduction to Logic and 
Scientific Method, p. 336. (Second edition.) New York: D. Appleton and Co., 1930. 
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and the elusiveness of verbal symbols. Aside from these diffi- 
culties, it should be said that the conventions of deductive logic 
are just as true to-day as ever, and if one accepts the fact 
that scientific inquiry is an inductive-deductive process, prin- 
ciples of logic are tremendously important. The student of the 
methodOlogy of educational research may find much that is 
archaic in deductive logic; he will also find much that is in- 
dispensable to good thinking. 

The appeal to scientific inquiry. Difficult as it is to fix an 
exact date for the beginning of the scientific method (because 
ideas do not come into being full grown), this event is assigned 
customarily to the seventeenth century.’® Galileo (1564-1642) 
made his famous experiment on the rate of acceleration of fall- 
ing bodies from the leaning tower of Pisa in 1589; he invented 
the telescope in 1611; Napier (1550-1617) published his work 
on logarithms in 1614; Briggs (1561-1631) used decimal nota- 
tion in 1617; Bacon (1561-1626) published his Novum 
Organum in 1620; Kepler (1571-1630) offered proof in 1609 
of the Copernican hypothesis that the earth and other planets 
move around the sun; Harvey (1578-1657) presented his trea- 
tise on the circulation of the blood in 1628; and Newton (1642- 
1727) furnished the mathematical proof of the hypothesis of 
universal gravitation in 1679.11 From these modest but ex- 
ceedingly important contributions has arisen the magnificent 
structure of modern science. 

Viewed historically, man’s first explanations of the natural 
phenomena about him were expressed in terms of the super- 
natural. This sort of explanation is well represented in the 
savage’s belief in magic, charms, witches, and other super- 
natural forces. Unable to understand the complexity of the 
natural environment in which he found himself, savage man 

10 Although Aristotle, who was intent upon seeking scientific explanations, emphasized 
the importance of proceeding from systematically collected facts (induction) to general- 
izations, these being the indispensable premises for deductive reasoning, in practice he 
failed because he did not see the real nature of induction and hypothetical thinking, 
and the necessity for sifting and probing facts. 

F. A. Westaway, Scientific Method: Its Philosophical Basis and Its Modes of Appli- 


cation, p. 76. (Third edition.) London: Blackie and Son, Ltd., 1924. 
11 Samuel Chester Parker, op. cit., pp. 113-9. 
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appealed to the supernatural. These explanations were recorded 
in the traditions of the tribe and interpreted by the tribal 
wise men. Gradually, as man emerged from savagery into civil- 
ization, he began to note many real and imagined relationships 
in nature, such as: wet feet cause colds; a circle around the 
moon portends bad weather ; one fish dropped into each hill of 
corn makes better corn; certain farm crops planted in a light 
moon grow better than when planted in a dark moon; fat men 
are more jovial than lean; individuals who learn rapidly forget 
rapidly; and physiognomical features are indicative of intelli- 
gence and criminal tendencies. All such unexplained uniformi- 
ties, imagined or real, growing out of the trial-and-error 
experiences of the race, are called empirical knowledge; thus, 
as a result, there are empirical agriculture, empirical medicine, 
and empirical education. Much that claims the name of educa- 
tion to-day belongs to the empirical category,’* based upon the 
trial-and-error efforts of the profession. Many of the so-called 
laws of the physical sciences are also empirical, that is, ob- 
served uniformities in nature for which no adequate reason or 
explanation can be assigned.*? One may know, for example, 
that certain drugs will produce certain effects, without know- 
ing the physiological processes involved ; or that certain chem- 
icals react in a given manner, without understanding why they 
do so, etc. Although man must frequently content himself with 
these empirical findings, the ultimate goal of science is com- 
plete explanation. The tremendous significance of the theory of 
evolution is to be found in the explanatory principle that it 
offers for so great a range of facts. Newton’s law of gravity, 
the nebular theory in astronomy, and the theory of relativity 
are other examples. We cannot, however, fully explain an ob- 
served uniformity in nature until we comprehend its numerous 
relations, until we discover the deeper nature of the underlying 
system, and until we understand why it acts as it does. When 
one is able to understand the uniformities of nature and to 
12 W. C. Trow, op. cit., pp. 19-21. 


13 J. Weiton, op. cit., p. 204. 
14 Daniel S. Robinson, op. cit., pp. 289-97. 
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offer satisfactory principles of explanation, one’s knowledge 
is complete, comprehensive, and scientific. This is the ultimate 
goal of all science. 

The nature of scientific thinking. The scientific method of 
research is an inductive-deductive mode of thinking through 
which one seeks to discover, establish, and explain the uni- 
formities of nature by appealing to experience.’® It involves a 
double movement of thought: first, a movement forward from 
the partially known and sometimes confused data of experi- 
ence, secured from all available sources (controlled systematic 
observation, uncontrolled incidental observation, previous in- 
vestigation, reading, conversation, and reflective thinking) 
toward universal relationships expressed as hypotheses, laws, 
principles, or propositions, that is, from the particular to the 
general; and second, a backward movement from these general 
propositions to the particular facts or data of experience, that 
is, from the general to the particular.’* Induction moves for- 
ward from the particular to the general. The mind begins with 
facts and moves in the direction of some interpretative prin- 
ciple, hypothesis, or explanatory concept." Deduction is a 
backward movement from the general to the particular. It aims 
at the verification and proof of hypotheses by bringing their 
logical implications into line with the facts of the case. Thus 
scientific thinking starts with facts and continually returns 
to facts to test and verify its hypotheses. 


15 There is a considerable division of opinion among logicians about the inductive- 
deductive character of the scientific method. Deductive logic is usually associated with 
syllogistic reasoning and inductive logic with scientific reasoning. Even the more formal 
logicians, however, appear to recognize a deductive element in the inductive methods of 
science: Thomas Fowler, The Elements of Inductive Logic (Third Edition. Oxford: The 
Clarendon Press, 1876); James Creighton, An Introductory Logic (Fourth Edition, re- 
vised. New York: The Macmillan Co., 1920); W. Stanley Jevons, The Principles of 
Science: A Treatise on Logic and Scientific Method (Second Edition. London: Macmillan 
and Co., Ltd., 1924); Daniel S. Robinson, The Principles of Reasoning: An Introduction 
to Logic and the Scientific Method (Second Edition, New York: D. Appleton and Co., 
1930); J. Welton, The Logical Bases of Education (London: Macmillan and Co., Ltd., 
1919), and Its Modes of Application (Third Edition. London; Blackie and Son, Ltd., 
1924). 

16 John Dewey, How We Think, pp. 68-78. Boston; D. C. Heath and Co., 1910. 

Matthew Thompson McClure, An Introduction to the Logic of Reflection, pp. 349-86. 
New York: Henry Holt and Co., 1925. 

17 Ibid., p. 362 ff. 
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Dewey supplies the following example of the inductive- 
deductive character of thinking: A man who has left his rooms 
in order finds them upon his return in a state of confusion, 
articles being scattered at random. The notion comes to his 
mind that burglary would account for the disorder. He has not 
seen the burglars, but this hypothesis offers a plausible expla- 
nation of the facts. It also occurs to him that his children are 
mischievous; they may have thrown the things about. Thus a 
competing hypothesis arises, but judgment is suspended until 
positive proof is had. At this point the deductive movement 
begins. Further observations, recollections, and reasonings are 
conducted on the basis of the suggested hypotheses. If bur- 
glars were responsible, certain things would have happened; 
articles of value would be missing. In this thinking process the 
man goes from the general to the particular, not merely, how- 
ever, to the original particulars (which would be fruitless or 
would take him in a circle), but to new facts, the actual dis- 
covery or non-discovery of which will test the validity of the 
supposed explanation. He searches for new evidence. He thus 
moves back and forth from facts to principles, and from prin- 
ciples to facts until a satisfactory explanation is derived, or 
the solution of his problem given up as a bad job.1® The 
scientific mode of thought involves operations similar to those 
described above, but more systematically performed. 

A second example may be taken from the field of education: 
from the partial data now available, it appears that the sub- 
ject-matter of the formally taught, logically organized school 
subjects may be less well retained than knowledge learned 
incidentally or functionally in natural life situations. One has 
observed that the rules of grammar, the facts of history, and 
the formulae of mathematics are almost always soon forgotten; 
but when one learns to skate, swim, or drive an automobile, 
one seldom if ever forgets. Thus, there is first a movement 
from the concrete facts of experience toward a working 
hypothesis. It appears that there is an observed difference in 


18 John Dewey, op. cit., pp. 82-3. 
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the products from these two methods of learning, and this 
difference is temporarily attributed to the mode of learning. 
At this point the process is reversed, and the movement from 
the general to the particular begins. The general proposition 
may now be compared with new facts systematically collected 
for the occasion. The general proposition may now take the 
form of a series of specific questions, such as: (a) Is it true 
that facts functionally learned are longest retained? (b) 
Does this difference obtain for all sorts of materials? (c) Is it 
true for individuals of all levels of training, maturity, and in- 
telligence? (d) If it is true that functionally learned subject- 
matter is longest retained, is the difference attributable to 
functional learning? (e) Can the difference be accounted for on 
the basis of repetition? (f) Can the difference be accounted for 
on the basis of drive or interest? (g) Can the difference be ac- 
counted for on the basis of meaning? etc. Each of these special 
suppositions will need to be carefully and systematically tested 
in the light of the already available facts and newly collected 
facts. As the thinking progresses, certain suppositions will need 
to be discarded, and new ones will take their places, until a 
defensible explanation is derived. The scientific worker starts 
with facts and continually returns to facts until his tentatively 
announced suppositions are fully tested and verified. 

Some distinguishing characteristics of the scientific 
method. Of the many distinguishing features of scientific 
thinking, reference may be made to six: (x) science is based 
on facts; (2) science employs the principles of analysis as a 
fundamental procedure in the comprehension of complex phe- 
nomena; (3) science employs hypotheses in the thinking in- 
volved; (4) scientific thinking is characterized by freedom 
from emotional bias; (5) science utilizes accurate measure- 
ment; and (6) science employs quantitative methods in the 
treatment of its data. To secure a better conception of the 
scientific method of investigation, each of these six distinguish- 
ing features will be discussed briefly. 

Science is based on facts. The greatest discovery ever made 
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in human thinking was that the way to determine whether a 
thing is present is to look and see.*® Galileo’s experiments with 
falling bodies at the leaning tower of Pisa represented not 
merely progress in his chosen science but a revolution in think- 
ing. It occurred to Galileo, following many centuries of theo- 
rizing about the problem, that the easiest way in which to settle 
the argument about the rate of acceleration of falling bodies 
was to look and see. A similar procedure occurred in the field 
of education when J. M. Rice *° suggested, not many years ago, 
that teachers find out whether pupils could spell by giving 
them achievement tests of spelling. There are innumerable op- 
portunities to-day in the field of education for the application 
of this principle of evidence. Too much argumentation centers 
about things that might be settled upon the application of the 
simple principle of “look and see.” As has been said, one of 
the distinguishing characteristics of the scientific method is its 
appeal to facts. 

Science employs the method of analysis in the comprehen- 
sion of complex phenomena. The natural phenomena with 
which man comes into contact are exceedingly complex. This 
is particularly true of the subject-matter of education. Man’s 
abilities to perceive and comprehend are, moreover, exceed- 
ingly limited. If he desires to succeed in his thought processes, 
he must not attempt wholes beyond his comprehension. The 
most difficult problems of life can be better comprehended by 
breaking them up into parts and by attending to each part 
separately. Instead of attempting to comprehend, for example, 
the whole of mental discipline at one bold stroke, it would 
seem better to think first of one of its parts: (1) Do habits 
of neatness acquired in handwriting transfer to other simple 
forms of behavior? (2) Does the ability to think logically in 
geometry transfer to the logical arrangement of the facts in a 
case of law? (3) Does transfer take place more readily with 

19R. D. Carmichael, The Logic of Discovery, pp. 30-1. London: Open Court 
Publishing Co., 1930. 


20 G, M. Ruch, The Objective or New-type Examination: An Introduction to Educa- 
tional Measurement, p. 12. Chicago: Scott, Foresman and Co., 1929. 
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readily as the application of a new instrument of measurement. 
The science of heat may be said to have begun with the con- 
struction of the thermometer; chemistry has been created 
chiefly by the careful use of the balance; and astronomy by 
the invention of the telescope. Progress in the scientific study 
of education has been closely associated with the development 
of accurate instruments of measurement. 

The use of quantitative methods in the treatment of data. A 
final distinguishing characteristic of the scientific method is 
its application of quantitative methods to the treatment of 
data. It has already been said that instead of relying upon 
estimates, guesses, and general impressions, the scientist meas- 
ures the phenomena under consideration as accurately as 
possible. The results of his measurements are, therefore, re- 
corded in comparable units that may be counted, added, sub- 
tracted, multiplied, and divided. Instead of relying upon the 
treacherous verbalism of ordinary language, the scientist em- 
ploys the new and more exact language of mathematics to 
make the analysis, classification, and summarization of data 
more reliable. Beginning with the careful study of one aspect 
of phenomena at a time, he assembles the pertinent data into 
meaningful categories and summarizes the facts into mathe- 
matical values, such as, means, medians, modes, probable 
errors, quartile deviations, standard deviations, and coefficients 
of correlation, which are less ambiguous than the terms of 
ordinary language. The careful measurement of phenomena 
and the mathematical treatment of the data thus collected are 
indispensable features of the scientific method. 

Steps in the process of scientific thinking. A further char- 
acterization of the general methodology of scientific research 
can best result from a discussion of the logical steps involved 
in scientific thinking. Briefly they may be stated as follows: *° 


x. A preliminary observation of facts 
2. The formulation of a bypothesis 


29J. Welton, The Logical Bases of Education, p. 137. London: Macmillan and Co. 
Ltd., 1919. 
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3. The testing of the hypothesis by comparison of its consequences 
with the results of a careful analysis of the phenomena under con- 
sideration 


A more detailed enumeration of the steps involved in scientific 
thinking may be stated as follows: 


1. The location and definition of a problem 

2. Survey of past experiences with the problem, previous investiga- 
tions, and the already available data to get ideas about past and pos- 
sible future solutions and methods of investigation 

3. The formulation of a hypothesis (or hypotheses) representing 
a tentative solution of the problem under investigation, to be em- 
ployed as a guide in the collection of additional data (by step 2 or 
step 5) which may lead to an accepted solution of the problem or to 
the formulation of a new hypothesis (or hypotheses) that may be 
employed in the collection of more data, etc. 

4. The mental elaboration of the hypothesis (or hypotheses), 
checking for agreement with fact, verifiability, and logical consistency 
(The mental elaborations here referred to may end in a belief that 
the solution is correct or in the formulation of new hypotheses to 
guide in the collection of new data, etc.) 

5. The collection of additional data (if necessary) through a new 
appeal to experience by means of measurement, observation, and ex- 
perimentation (The already available data may be incomplete, or 
collected under conditions that throw doubt upon their trustworthi- 
ness, or recorded in terms that are ambiguous and subject to many 
interpretations or mis-interpretations, thus making it necessary to col- 
lect new data.) 

6. The analysis, classification, and summarization of the data col- 
lected 

7. The formulation of new generalizations representing observed 
uniformities, explanatory principles, or scientific law 


The last step, that is, the formulation of explanatory prin- 


“ciples or scientific laws, usually follows only upon the projec- 


tion and systematic testing of many hypotheses. A single in- 
vestigation is seldom sufficient. If one is desirous of undet- 
standing the underlying system of complex phenomena such as 
are observed in professional education, he should expect to: 
spend many hours in the investigation of these phenomena ; in 
collecting, analyzing, and systematizing data; and in reflecting 
upon the results. It is only by the persistent, continuous, ard 
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systematic investigation of a limited field of research that one 
may expect to attain scientific insight into the more complex 
problems of education. 

It may appear from the list of steps given above that scien- 
tific thinking follows some invariable sequence. Such, however, 
is not the case. For example, many problems, new and un- 
foreseen, will arise in the process of investigation. It has some- 
times been said that the best way to develop a problem 
consciousness is to experiment. New hypotheses or altered 
hypotheses may arise at almost any point in the process of 
investigation. These lead to the collection of new data, new 
reasoning, and new research. Reference has already been made 
in this chapter to the backward and forward movement of 
scientific thinking. One may go from step two to step three 
and the reverse many times before going to step four, and if 
the data are reasonably complete and accurate, one may never 
proceed beyond step four, feeling that the solution has with- 
out doubt been reached or that it is reasonably satisfactory. 
Such is the case in many of the simpler instances of thinking 
about everyday problems of life and in philosophical research, 
as will be discussed later. One proceeds to step five only when 
new and more accurate data are needed. 

The succeeding sections of this chapter will be devoted to a 
brief discussion of the seven rather detailed steps in scientific 
thinking enumerated above. 

The location and definition of a problem. The first step in 
problem solving is the location of a problem, a felt need, or 
a recognized difficulty. Dewey cites three sorts of situations 
leading to difficulties.** The first arises out of the lack of 
adaptation of means to end. One desires to keep an engagement 
at a certain time, and the existing hour taken in connection 


30 The terms “already available data’? and “new data,” because of the different inter- 
pretations placed upon them, have led to much confusion. The data may be new to 
the investigator, but if they have been experienced and recorded by others, whatever 
their form or source, they are “already available data.” If the data involve new ob- 
servations, measurements, and records, even though similar observations may have been 
made by previous investigators, we refer to them as “new data.” 

$2 Tohn Dewey. W. ^it. pD. 72-3. 
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with the location, which may be some distance away, is in- 
congruous. The given conditions cannot themselves be altered; 
the problem is “the discovery of intervening terms which when 
inserted between the remoter end and the given means will 
harmonize them with each other.” In this case the difficulty 
resides in the conflict between conditions at hand and a de- 
sired result. Much of the search for better means to an end, 
in education, falls within this category. The second situation 
leading to difficulties arises in identifying the character of an 
object. In this case some temporarily entertained belief is 
incompatible with certain observed facts. Dewey uses the ex- 
ample of the identification of what appeared to be a flagpole 
on a ferry boat. He says, suppose we symbolize the qualities 
that suggest flagpole by the letters a, b, c, and those that 
oppose this suggestion by the letters p, q, 7. Let us further as- 
sume that the qualities themselves are really not inconsistent, 
but in the absence of other data they appear to conflict and 
thus lead the mind to different and incongruous conclusions— 
hence the problem. Just as in our first case, the object here is 
to discover some intervening term or terms, symbolized by d, 
g, L and o, which, when combined with the existing terms, will 
form a single whole. The method again is that of discovering 
the necessary intervening term or terms. The third situation 
leading to difficulties arises in explaining an unexpected event. 
In this case, an observer trained to observe natural laws or 
uniformities finds something odd or exceptional in the behavior 
of some natural phenomenon. The problem is to reduce an 
apparent irregularity to rule, law, or type. Here the method of 
solution is also to seek for an intermediary term or terms which 
will connect, by regular linkage, the seemingly odd or excep- 
tional behavior with conditions known to follow from 
processes supposed to be operative. This feeling of need, dis- 
crepancy, or difficulty is defined as a problem. 

Having experienced a difficulty, the next step in scientific 
thinking is to define the difficulty. In complex situations the 
exact nature of a difficulty is not always immediately under- 
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stood. The situation demands careful observation, deliberation, 
and analysis. To accept the first suggestion that presents itself 
is to be unscientific in one’s thinking. It is into just this error 
that untrained workers are likely to fall. In the case of a physi- 
cian this careful analysis of a complicated situation before 
passing judgment is called diagnosis. The essence of this process 
is that one attempts to determine the nature of a problem 
before proceeding into its solution. It is this, more than any- 
thing else, that distinguishes uncritical and uncontrolled think- 
ing from verified knowledge.*? The selection and definition of 
a problem are discussed at greater length in Chapter IT. 

The survey of already available data. One seldom begins re- 
search upon an absolutely new problem. As a rule, one has had 
prior contact with the problem and a certain amount of experi- 
ence with it. This earlier experience furnishes a certain back- 
ground of information useful in defining the problem and in 
formulating hypotheses. Not only has the individual probably 
had prior contact with the problem himself, but other persons 
have probably come upon the same difficulty ; they may have 
studied the problem systematically; and they may have re- 
ported the results of their thinking in published articles, mono- 
graphs, or books. As a matter of fact, the student of educa- 
tional problems finds the whole of man’s past published ex- 
periences (and such unpublished experiences as he can learn) 
with such phenomena available for use in defining his prob- 
lems, in the formulation of hypotheses, and in deciding 
whether additional data are necessary. 

It has been said that the original mind is the well-informed 
mind. One secures ideas about what is worthy of investigation, 
of methods of investigation, and of the probable solutions to 
problems by being well informed about such things. There is 
a large amount of useless repetition of investigations in the 
field of education as a result of ignorance on the part of re- 
search workers concerning previous studies. Although fre- 
quently the results of previous investigations need to be 


82 Tohn Dewey, op. cit., pp. 73-A. 
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verified by further research, this should be intelligent research 
in which the worker is thoroughly informed about what has 
preceded him. The collection of new data according to the 
conventions of science is a time-consuming activity and should 
be undertaken only after the research worker has assured him- 
self that the problem is worthy of the special effort to be 
expended upon it and has not already been solved. This prob- 
lem is discussed in considerable detail in Chapter TI. 

Before leaving this subject of the available data, emphasis 
should be placed on two facts. In the first place, the use of 
past experiences with phenomena is an appeal to history, and 
historians have developed a system of working with documen- 
tary evidence with which research workers should be familiar. 
It is true that one can “muddle through” a survey of past 
thinking upon some subject without knowing much about the’ 
methods of historical research, but training in this field is 
essential to productive scholarship. So important is the use of 
the historical method in educational research that a separate 
chapter of this book has been devoted to it (Chapter VI). In 
the second place, the writers wish to emphasize the importance 
of historical analyses of past experiences with educational 
phenomena by stating that one may devote a lifetime of re- 
search to this field alone. One may concern himself, for ex- 
ample, with the history of selected aspects of the school’s 
program of instruction, such as school support, the nature of 
the curriculum, or changing concepts of method; with selected 
problems and issues, such as the school-subject organization 
yersus the activity organization of subject-matter, the whole 
versus the part methods of instruction, or formal discipline; 
or with general surveys of elementary education, secondary 
education, or higher education. The same point might be made 
with reference to concentration on or specialization in other 
steps in a complete act of thinking. When a worker chvoses 
to make the careful collection and systematization of past ex- 
periences with educational phenomena his special province of 
research, he becomes an educational historian. 


20 EDUCATIONAL RESEARCH 


The formulation of hypotheses. Earlier in this chapter the 
use of hypotheses in thinking was discussed as a distinguishing 
feature of the scientific method. Having come upon some diffi- 
culty or some problem, one normally begins to cast about for 
a solution. If the difficulty is one frequently experienced be- 
fore, one may recall the solution or probable solutions, and the 
process may be greatly shortened. This is the case with a 
fountain pen. At certain fairly definite intervals the pen ceases 
to write. It is a real economy in thinking to remember that 
the pen is out of ink again and to eliminate the necessity of 
rediscovering each time the reason for the pen’s failure to write. 
In educational research, however, we are generally not so much 
interested in this sort of hypothetical thinking as we are in the 
discovery of new relationships. With this thought in mind, the 
question naturally arises: “How does one get ideas about the 
probable solution to difficulties?” It has already been said that 
the original mind is the well-informed mind. One purpose of 
the survey of one’s own experiences and of those of others 
with given problems is to secure ideas concerning probable solu- 
tions. One might ask further: “Are there not methods for 
facilitating the formulation of hypotheses?” “Are there not 
some habits of thinking which are more conducive than others 
to the discovery of hypotheses?” etc. These and similar ques- 
tions will be discussed in Chapter IV on the formulation and 
logical development of hypotheses. 

The logical development of hypotheses. One of the character- 
istics of a good hypothesis is its amenability to deductive 
reasoning. In this connection there are two very significant 
applications of the deductive technique. In the first place, 
hypotheses may be checked for logical consistency as infor- 
mally or as formally as one chooses. If the checking is done for- 
mally and systematically, one will doubtless employ a large 
share of what has been learned in the study of syllogistic rea- 
soning and deductive logic. A second process that one may use, 
and ordinarily does, at this point in the testing of hypotheses is 
to study the agreement of their deduced consequences with the 
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already known or readily discoverable facts in the case. One 
might say, for example, “If this tentative supposition is true, 
then such and such things should follow;” or “If this hy- 
pothesis is not true, then such and such other things should 
follow ;” etc. Thus, when the professor mentioned by Dewey 
returned to his room to find it in utter confusion, he formed 
some kind of a tentative conjecture as to the probable cause, 
The idea of burglary may have suggested itself. From previous 
experience he has already formed some concept of burglary. 
With this concept in mind, he tests the burglary hypothesis by 
applying his knowledge of burglary to the facts in this par- 
ticular situation. It is a process of checking back and fort! 
from the facts in a case to a hypothesis and from the hypothesis@ 
to the facts in the case.** Dewey "‘ refers to this step as the; 
rational elaboration of an idea, as the process of developing itso" 
bearing and implications. Creighton ** emphasizes the fact thatimt 
a hypothesis from which nothing can be deduced is of no? 
value. Conjectures that seem plausible at first sight are often; 
found unjustified or even absurd when their full consequences 
are traced. Even when reasoning the bearings of a false suppo- 
sition does not lead to its rejection, such a procedure develops 
the main idea and makes it more amenable to actual verifica- 
tion. 

The collection of new data. Let us repeat that science pro- 
ceeds from particular facts to hypotheses, and from hypotheses 
to facts. If the data already available in the documents and 
the memories of man are adequate for the solution of a prob- 
lem, no new appeal to experience will be necessary. In such 
cases the completion of the first four of the seven steps in the 
complete act of thinking furnish, at least temporarily, the 
terminal point of the search for truth. In many instances, how- 
ever, the record of past experiences may be incomplete or 
may have been made under conditions that cast doubt upon 
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their trustworthiness. For example, when the data have been 
collected by biased or unseeing observers, when the means of 
collecting the data have not been satisfactorily validated, when 
the observations ‘have not been systematic and controlled, and 
when the records are made in the ambiguous terms which 
characterize the ordinary verbal record, it may be necessary 
to make a new appeal to experience. 

The new appeal to experience may involve a simple noting 
of the presence or absence of simple units of a phenomenon, 
such as, the number of births, deaths, marriages, divorces, etc., 
or it may involve the application of more refined instruments 
of measurement such as standardized tests, rating scales, and 
mechanical instruments of measurement, according to the con- 
ventions of science. Further, the collection of data may be 
carried on by either controlled or uncontrolled observation. 
It is sometimes advantageous to study phenomena under the 
artificially controlled conditions of experimentation.*° To ex- 
periment is to place phenomena, by means of artificial controls, 
in a favorable location for observation. Thus, the collection of 
data may involve simple counting, or simple counting plus 
measurement, under controlled or uncontrolled conditions of 
observation. A series of subsequent chapters will deal with this 
topic, 

The analysis, classification, and summarization, The fact 
that analysis is one of the distinguishing characteristics of 
the scientific method has already been emphasized. From one 
point of view the topic of analysis belongs more appropriately 
under the heading of collection of data than with classification 
or summarization of data. Some analysis of a phenomenon, and 
the circumstances surrounding it, into elements, constituents, 
or factors must precede any attempt to control the given fac- 
tors for experimental purposes, Though this is true, analysis is 
also an important phase of the classification and summariza- 
tion of data. In the classification of data one assembles data 


36 Thomas Fowler, op. cit, pp. 38-50. 
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of like characteristics.’ A chair, a table, a couch, and a rug 
may all be thought of as household furniture and assigned to 
this category. They are also manufactured articles and may 
be classified as such. The types of categories constructed in the 
classification of data will depend, of course, upon the phe- 
nomena under investigation, and the purposes of the research in 
progress. The contribution made by the summarization of data 
to scientific thinking is the economy of thinking brought about 
by the assemblage of a multitude of facts into a single whole. 
If one collects a large number of data about some phenomenon, 
it becomes correspondingly difficult to retain them in mind 
as the number of facts increases. For example, instead of at- 
tempting to retain the individual I.Q.’s of 300 elementary- 
school children, it ordinarily serves one’s purpose better to 
calculate the mean I.Q. Likewise, one might calculate through 
the use of appropriate statistical methods the spread of talent, 
the relationship of one trait to another, etc. The special 
economies arising out of such treatments of the data have 
given rise to the subject of educational statistics. Before one 
progresses very far in the scientific study of the problems of 
education, one must have mastered at least the rudiments of 
statistics. It is a major function of statistics to furnish re- 
search workers with economical and effective means of analyz- 
ing, classifying, and summarizing data. This problem is 
discussed in greater detail in Chapter XI. 

Generalization. The ultimate goal of science is the formula- 
tion of valid generalizations in the form of verified supposi- 
tions, explanatory principles, or scientific laws.** If the data 
are complete, convincing, and important, the generalization 
may take the form of a complete scientific explanation. Most. 
investigations will fall short, however, of this ultimate goal 
of science, contributing merely to the formulation of new hy- 
potheses to be employed in the search for new insight. We 
have already shown that the problems of education are com- 


87 Thomas Fowler, op. cit, pp. 50-87. 
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plex; it is only through the careful, painstaking, and systematic 
investigation of such problems that one may hope to obtain 
valid solutions possible of statement in the form of generali- 
zations. Chapter XII discusses at greater length the problem 
of formulating conclusions and generalizations. 

Science, history, and philosophy in research. Before bring- 
ing this discussion of the nature of scientific thinking to a 
close, it is essential to comment briefly on the supplementary 
character of history, science, and philosophy in educational 
research. One may, if he chooses, follow through, with refer- 
ence to some particular problem, the entire sequence of steps 
of a complete act of thinking, as set forth in the foregoing 
discussion, or may specialize in a single phase of the research 
process. If one specializes, for example, in systematizing the 
records of past experiences with educational phenomena, we 
call him an educational historian. If one specializes in the 
critical examination of educational theories, hypotheses, and 
generalizations in the light of data which are already avail- 
able, we call him an educational philosopher. If one specializes 
in the solving of educational problems by making new appeals 
to experience through systematic, controlled and uncontrolled 
observation, in field or laboratory, we call him an educational 
scientist in the classical sense of the term. These several ap- 
proaches to research are merely phases of the complete act of 
thinking, and there can be no fundamental conflict between 
them. 

Concluding statement. The authors have presented in this 
chapter a discussion of the nature of scientific thinking: first, 
the sources of evidence, namely, personal experience, custom 
and tradition, authority, syllogistic reasoning, and scientific 
inquiry; and second, the three levels of knowledge: super- 
natural beliefs, empirical findings, and scientific explana- 
tions. The scientific method was defined as an inductive- 
deductive mode of thinking in which one seeks to explain 
the uniformities of nature by appealing to experience. It 
possesses six distingvishing features: (1) its appeal to facts, 
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(2) its application of the method of analysis, (3) its use of 
hypotheses, (4) its freedom from emotional bias, (5) its use 
of objective measurement, and (6) its use of quantitative 
methods in the treatment of data. Seven steps were recognized 
and defined as constituting scientific procedure: (1) the loca- 
tion and definition of a problem, (2) a survey of the already 
available data, (3) the formulation of a hypothesis, (4) the 
development of the hypothesis, (5) the collection of new or 
additional data, (6) the analysis, classification, and summari- 
zation of data, and (7) generalization. Finally, a brief com- 
ment was offered concerning the supplementary character of 
science, philosophy, and history. The remaining chapters of 
this volume are organized roughly about these seven steps 
in scientific thinking. 


PROBLEMS AND EXERCISES * 


1. What are the sources of evidence (personal experience, custom 
and tradition, authority, syllogistic reasoning, or scientific inquiry) 
commonly used to-day by the majority of people in dealing with 
problems touching such fields as agriculture, politics, government, 
economics, sociology, psychology, and education? 

2. Select a research study of recent date in the field of education, 
and analyze it in terms of the six distinguishing features of scientific 
method and the seven steps in scientific procedure discussed in this 
chapter. 


*The problems at the end of the chapters are optional, and may be 
assigned at the discretion of the instructor. No attempt is made to prescribe 
instructional methods in dealing with the materials of this book, although 
teaching and learning procedures in applying principles of research are 
suggested or at least implied by the nature of the problems themselves. 
Of course, the student who is working on a thesis or some other type of 
investigation, from a functional point of view, has a very desirable type 
of problem to which the principles of research may be applied. There are 
various possible ways of using these exercises, as suggested below: 

1. A single problem selected: (a) for the entire class, (b) for each smal} 
group of three or four students, or (c) for each individual, carried through 
all or most of the chapters and exercises with appropriate applications in 
connection with each chapter, procedure, or step in research. 

2. A different problem selected in connection with each chapter, pro- 
cedure, or step in research: (a) for the entire class, (b) for each smal! 
group of three or four students, or (c) for each individual. 
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CHAPTER II 


THE SELECTION OF A PROBLEM: NEEDED RESEARCH 
IN EDUCATION 


Introduction. The purpose of this chapter is to aid educa- 
tional workers in identifying problems through an analysis of 
both their own areas of service and the various types of litera- 
ture concerned with completed research, trends, critiques, 
prophecies, investigations in progress, and needed research. 
Criteria for use in selecting a problem and suggestions for 
formulating, stating, defining, and delimiting the problem will 
be presented. 


THE DISCOVERY OF PROBLEMS 


Many unsolved problems in education. A too common be- 
lief of educational workers, whether in field service or the 
graduate school, is that most of the problems of education have 
been solved. They express amazement at the relatively large 
number (in excess of 3200 at this writing) of Masters’ and 
Doctors’ theses in education completed each year; from seven 
to eight hundred books, monographs, and bulletins published 
annually; and thousands of studies published in research 
periodicals as evidence that education must have few additional 
difficulties with which to cope. That this conception is utterly 
fallacious will be demonstrated in the present chapter. 

As long as the major task of the school is to educate youth, 
with a host of variable factors in terms of pupil, teacher, cur- 
riculum, administrative organization, community, etc., there 
always will be problems to solve. Human nature reduces itself 
to quantitative terms and to generalizations with much greater 
difficulty than do inanimate objects in such fields as physics 
and astronomy. As a matter of fact, it has been pointed out 
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that of our two parents, one has, let us say, 17,000 germ cells 
and the other 300,000,000,000 germ cells. The probability of 
the unique combination that produces a given child is one in 
5,000 million million. Although some biologists are more con- 
servative in their estimates, it is perfectly clear that every 
child is unique and has no duplicate in the past, present, or 
future. Even 2,000 “standard white rats” which grew along a 
normal growth curve of weight and size in Wistar Institute at 
Philadelphia were considered to possess individuality, although 
not so variable as human beings. 

Granted that the research worker in education is confronted 
by difficulties in the variables mentioned, he may well be 
stimulated to find more nearly exact answers to his problems 
by noting the accomplishments of measurement in certain other 
fields. Hubbard states these achievements in a striking way.’ 


Through measurements we learn of the proton, smallest component 
of the atom, of which 10,000 million million in a line would measure 
one inch. Through measurements we study island universes a hundred 
million light years away, a million light years across, receding from 
us with speeds as high as 10,000 miles a second. 

Men now measure the change in the rotation rate of the earth, a 
rhythmic increase and decrease over a range of 4o seconds in 80 
years, They can measure changes in the orbit of the moon caused by 
the shift of masses within the earth. They measure the movement 
of the north pole of the earth, 40 feet a year. Then, too, they can 
measure the brightness of the planet Saturn as affected by the change 
in sunlight. They even measure the change in the size of the sun 
as its pent-up energy increases in advance of the new cycle of sunspots, 

I have often held in my hand the extraordinary device by which 
we can measure the heat that the North Star may radiate on the 
finger-tip, a quantity so small that a million years of shining would 
just heat the finger-tip one degree if no heat were lost. The sensi- 
tivity of the modern electron tube may be represented by comparing 
it to the detection measurably,of two drops of water in the annual 
overflow of Niagara. Another delicate device perfected in Wisconsin 
records the footsteps of a man or those of a child walking on the 
ground a quarter of a mile away. 


1H. D. Hubbard, “Unmeasured Goals,” Baltimore Bulletin of Education, X (Marchi 
April, 1932), 161-7. 
2 Ibid. 


THE SELECTION OF A PROBLEM 33 


Although the appendix will list in some detail unsolved ‘prob- 
lems in the various subdivisions of education, this introductory 
statement may well enumerate pressing issues which confront 
the school after several years of economic depression.’ 

It should not be assumed from a review of such advances 
in science that most of the problems have been solved. As a 
matter of fact, with increased knowledge in a given field come 
many new problems pressing for solution. If a ball of light 
is likened to truth, and darkness to ignorance or unsolved 
problems, to increase the size of the sphere of light or to ex- 
pand the limits of knowledge really serves to increase the 
area of darkness or unsolved problems on the fringe of illumi- 
nation or truth. This statement may be substantiated by com- 
paring analyses of needed research in education, prepared a 
quarter of a century ago when the field was very limited, with 
the extensive current lists of needed investigations referred to 
in this chapter. In one of the early discussions * of needed re- 
search, Cubberley mentions finance, county reorganization, 
and units of cost; Dearborn and Thorndike stress experimenta- 
tion and quantitative measurement; while Monroe urges co- 
operation in research and shows the opportunity and need for 
research in educational history.’ 

When so much is said and written about “fads and frills” 
and a return to the “fundamentals” of the “good old days,” 
school people must know educational history in order to evalu- 
ate accurately the new and the old. For adequate analysis the 
problems of class size, individualized instruction, and progres- 
sive education require the contribution of the educational his- 
torian. When legislatures are passing drastic school laws, which 
inevitably must be changed when the members are saner, 
wiser, and less panicky, the efforts of the relatively small 


3 Carter V. Good, “Educational Research and the Times,” Journal of Educational 
Research, XXVII (October, 1933), 139-41. ; 

4E. P. Cubberley, W. F. Dearborn, Paul Monroe, and E. L. Thorndike, “Research 
within the Field of Education, Its Organization and Encouragement,” pp. 1-54. The 
School Review Monograph, Vol. I. Chicago: Department of Education, University of 
Chicago, 1911. A % 

5 Paul Monroe, “Opportunity and Need for Research Work in the History of Educa- 
tlon,” Pedagogical Seminary, XVIL (March, 1910), 54-62. 
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group of research workers in the field of school legislation 

_ should be extended more than ever before. Suggestions or com- 
mands for changes in curriculum, organization, or method, dic- 
tated by the voice of legislative or executive authority, born 
of necessity or panic, must be tested experimentally in actual 
school situations for any worth-while evaluation. There is 
more urgent need for surveying current school conditions in a 
time of economic depression than during a period of prosperity. 
In fact, the past few years have been remarkable in the 
number of great national surveys conducted (modern foreign 
languages, land-grant colleges, secondary education, teacher 
education, school finance, sociai studies, and the White House 
Conference on Child Health and Protection), although it is 
true that appropriations were made for some of these studies 
prior to 1929. Tests are needed that will measure the products 
of the “new education,” which are said to involve values other 
than factual information. It is desirable to know what effect 
present economic, social, and home conditions have on the ac- 
complishment of the child, as well as on his attitudes, per- 
sonality, character, and emotional life. This type of investiga- 
tion involves detailed case study of the children represented, 
as well as adequate instruments for measurement. 

Inability to recognize problems. The difficulty of many 
educational workers, especially beginners in the field and in 
the graduate school, is in recognizing the numerous problems 
which are obvious to those with the insight to sense such 
issues. The experienced research worker may be compared 
with the Indian or woodsman who grasps the significance of a 
faint haze on the distant horizon and of a broken twig or dis- 
turbed fallen leaf. The eye of the novice or city dweller may 
be physically just as keen as that of the trapper, but he fails 
because of limited experience to recognize the meanings in- 
volved. This characteristic inability of immature workers to 
see problems has been aptly described by Trow: ® 


6W. C. Trow, Scientific Method in Education, p. 40. Boston: Houghton Mifflin Co., 
1925. 
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Oftentimes students finish college and go on to graduate school 
before they discover that they need to do some original research 
before they will be awarded a graduate degree. So they confer with 
some member of the graduate faculty and ask him for a problem to 
work on; they can’t seem to think of any. Can’t think of any, and 
the world fairly bursting with them! x 


A more detailed account of the “problem blindness” of the 
graduate student was given by Buckingham when he was 
serving in the dual capacity of college professor and editorial 
writer. A student has entered the professor’s office, and after 
a preliminary remark or two, the following conversation en- 
sues: 7 


“Tve got to write a Master’s thesis,” says he, “and Pd like to talk 
to you about a topic.” The statement ends with a slight upward in- 
flection as if, in spite of its grammatical form, a sort of question were 
implied. After an awkward pause Mr. Blank (the student) repeats that 
he would like to talk about a thesis topic. Whereupon the editor (and 
professor) suggests that he go ahead and do so. 

It transpires, however, that the editor-professor has misconceived 
Mr. Blank’s meaning. He has no topic to talk about. In fact, instead 
of coming with a topic, he has come to get one. He looks so expect- 
ant, too; purely, as one might say, in a receptive mood. 

No, he has no problems to suggest. He gives one the impression 
of having just learned about this thesis business, and of being entirely 
open-minded on the subject. At least, one gathers that he has no bias 
toward any particular topic and certainly no preconceived notions, 
[Graduate professors will recognize this as a familiar situation.] 

A conversation ensues. The editor—playing for the nonce his pro- 
fessorial réle—asks in what department Mr. Blank is majoring, what 
courses he has taken, what positions he has held, and for what type 
of educational service he is fitting himself. At one stage of the result- 
ing exchange of ideas Mr. Blank brightens. With some modesty, yet 
with the undeniable air of a discoverer, he suggests that he might 
correlate intelligence and achievement in the high school. He could 
give some tests in the school with which he is connected; and his 
friend, the principal of the X school, would probably let him give 
some tests there; and maybe he could get one or two more schools 
ii he stopped to think about the matter. And, O yes! how many 
schools does the professor think would be needed to get results that 
you could depend on? On being told that intelligence and achievement 


7*The Editor Turns Professor.” Educational Research Bulletin. VI (September 14, 
1927), 252-3. 
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—so far as either is now measurable—have already been correlated by 
hundreds of people, Mr. Blank helplessly withdraws within himself, a 
discouraged seeker after truth in a world where all the problems 
have been solved. 


However, such statements are in no sense to be interpreted 
to mean that all field workers and graduate students are “prob- 
lem blind.” It is true that almost daily many educational 
workers in service encounter practical and immediate problems 
which must be attacked and solved, at least empirically. There- 
fore, the limitations of a considerable number of such teach- 
ers, supervisors, and administrators may be not so much a lack 
of problem consciousness as a failure to deal with their prob- 
lems scientifically. For that matter, even though the problem 
is recognized and the solution is being*actively sought, the 
trained worker may fail to see the answer. After reading the 
first description of Bell’s telephone, Moses G. Farmer said 
that the thing had flaunted itself in his face a dozen times 
within the past ten years, but he had been too blind to see it.* 
Too many workers fail to see possible improvement in an exist- 
ing situation. It was said in 1900 by one in authority that the 
incandescent lamp was so perfect a device that it could never 
be much improved. At that time it was only about one per 
cent efficient in terms of an ideal light, and since then has been 
rendered five times as efficient, with much improvement still 
possible.® 

Discovering problems. Suggestions have been made for 
identifying experimental problems, which may also be applied 
in locating issues susceptible of attack by other research tech- 
niques: (1) by becoming a scholar in one or more specialties 
as early as possible; (2) by reading, listening, and working 
critically and reflectively; (3) by considering every obstacle 
an opportunity for the exercise of ingenuity instead of an in- 
superable barrier; (4) by starting an investigation and watch- 
ing problems bud from it; and (5) by remembering those 


8T. A. Boyd, Research, p. 274. New York: D. Appleton-Century Co., 1935. 
9 Ibid., pp. 218-6 
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problems already found, that is, by keeping a systematic record 
of original ideas and problems.’° 
Each of the five overlapping methods of identifying prob- 
lems mentioned by McCall has possibilities, as applied to the 
work of the field or university investigator. The first two sug- 
gestions are concerned primarily with the professional or 
specialized training of the worker and with his knowledge of 
the literature dealing with progress made and anticipated. 
The first of these problems is discussed in Chapter XV, while 
the second is treated in Chapter III and in the present chapter. 
The last three items are of concern in this chapter, and deal 
with the recognition of problems which present themselves 
(already discussed), systematic recording of such difficulties, 
and with related issues which frequently appear during the 
process of experimentation or investigation. 

Probably the most concrete suggestions for discovering and 
recognizing problems which can be offered in this chapter are 
(to analyze the available educational literature on the given 
area or subject in terms of: (1) distribution of research com- 
pleted in the various fields of education, (2) authoritative 
statements of accomplishment and completed research, (3) 
analyses of trends, (4) illustrative critiques of given fields, 
(5) educational prophecies, (6) research under way, and (7) 
needed investigation. Familiarity with this type of literature 
should be of equal value in locating the problems of education 
in general and in analyzing one’s individual difficulties. Al- 
though these analyses are concerned primarily with the prob- 
lems of teachers and supervisory officers, for the sake of com- 
pleteness and for the benefit of the large group of principals 
and superintendents who must investigate administrative 
problems by way of promoting more effective instructional 
adjustments, selected discussions of the status of research in 
school organization and administration are included, The state- 
ments of research completed or needed have been analyzed, 


10W. A. McCall, How to Experiment in Education, pp. 7-8. New York: The Mac- 
millan Co., 1923. 
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following the given author’s own organization, whether in 
terms of school subjects, levels of instruction, or more spe- 
cific and limited educational problems. 


PROBLEMS EMPHASIZED IN EDUCATIONAL LITERATURE 


1. Frequency of investigation of various problems of 
education. Some indication of problems investigated fre- 
quently in the past is provided by analyses of the studies 
which have been published over a given period of years. In 
Tables I and II are the results of Franke and Davis’ study of 
the amount of research produced in the various fields and at 
the several levels of education respectively, as reported in 
magazines. Only data for 1920-1929 are reproduced." 


TABLE I 


DISTRIBUTION OF THE AMOUNT OF RESEARCH PRODUCED IN THE VARIOUS 
FIELDS OF EDUCATION DURING 1920-1929 


Articles Pages 
Field 
Number | Per Cent | Number | Per Cent 

Administration Ap iah ssia «We wal 233 11.65 1991 10.32 
Curriculum and course of study...... 161 8.05 1642 8.50 
AMUSO Sot SmI EN An sien Fis eles EASA 189 9-45 1728 8.95 
Child psychology..........-+..+++- 102 5.10 1321 6.84 
Educational psychology.........--.. 140 7.00 1572 8.14 
Educational measurement and sta- 

LIStICR ee un buat Seta E Aa EA eects 844 42.20 8120 42.03 
Guidance. . 119 5.95 907 4.70 
Character. . 72 3-60. 735 3.81 
Physical education... 15 “75 83 43 
Teacher training 125 6.25 1212 6.28 

Totals. 3 scandens sheers ie 2000 | 100 I9Q3II | 100 


In commenting on the fields of investigation represented in 
Table I the authors show that since 1890: (1) a relatively 


11'Paul R. Franke and Robert A. Davis, “Changing Tendencies in Educational Re- 
search,” Journal of Educational Research, XXIII (February, 1931), 133-45. 
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large amount of emphasis has been given to administration, 
child psychology, and educational measurements and statis- 
tics; (2) child psychology has held its most prominent position 
from 1890 to 1899, educational administration from 1900 to 
1909, and educational measurements and statistics from 1910 
to the present time, showing an increase from 1920 to 1929. 


TABLE I 


DISTRIBUTION OF THE AMOUNT OF RESEARCH PRODUCED AT THE VARIOUS 
LEVELS OF EDUCATION DURING 1920-1929 


Articles Pages 
Level 
Number | Per Cent | Number | Per Cent 
Elementary: tie wen aur i S 597 29.85 6294 32.59 
Elementary-secondary...........--- 12 6.2 817 4.23 
Secondary msie acas sus veins eerie 520 26.0 4935 25.56 
Secondary—higher..........-.+.+-- 44 2.2 415 2,15 
Higher). Jone arenas arenes 715 35-75 6850 35:47 
‘Totals, ace euee dean denote, 2000 I00 IQ3II 100 


With respect to levels investigated they state that: (1) re- 
search in elementary education has held a conspicuous posi- 
tion from 1890 to the present time; (2) there is a compara- 
tively small amount of research on the elementary-secondary 
level irrespective of year-periods, the greater amount of em: 
phasis having been placed on the secondary level rather than 
on the combination, elementary and secondary; (3) there is a 
negligible amount of research on the secondary-higher level ; 
and (4) there is a relatively great amount of research devoted 
to higher education throughout the four periods. 

A classification of fields represented in the entire group of 
Doctors’ theses in education from 1918-1931 is reported in 
Table III? 

The data of Table III may be interpreted briefly. In view 
of the fact that many graduate students have been, and are, 


12 Carter V. Good, “Fields and Types of Research in Education, 1918-31,” Journa 
Í Educational Research, XXIV (June, 1931), 33-43- 
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school administrators and supervisors, it is not surprising that 
a large number of theses completed, and under way in 1930-31, 
were in the field of administration and supervision. A com- 
parison of the total number of completed theses in psychology 
and measurement with those under way in 1930-31 indicates 
a relative decrease in this field; this may be a natural reac- 
tion to the overemphasis ‘on tests and measurements during 
the decade following the war period. 

It is to be expected that an investigator in the field of 
method will conduct his study in a given school subject, which 
explains the large number of theses under the heading, “School ` 
subjects,’ as compared with the smaller number under “Gen- 
eral method and theory.” Although Table III does not present 
such information in detail, the investigator was impressed by 
the total absence of Doctors’ theses under way in art educa- 
tion in 1930-31 and by the small numbers in certain other 
school subjects: commercial education, three; foreign lan- 
guages, three; home economics, seven; manual, industrial, and 
agricultural arts, eleven; and music, four. The number of 
theses under way in other school-subject fields in 1930-31 was 
as follows: English, reading, and language, thirty-nine; mathe- 
matics and arithmetic, thirty-four; moral, religious, and char- 
acter education, forty-eight; physical and health education, 
twenty-one; science, thirty; and social studies, twenty-two. 
The recent interest in problems of moral, religious, and char- 
acter education is evidenced by the comparatively large num- 
ber of studies under way in this field. The number is large, 
however, partly because certain schools of education classify 
theses in religious education under the heading of education. 

There is a decrease in the number of theses dealing with 
philosophical or subjective aspects of educational theory, prob- 
ably caused by wider use of the objective methods and tools 
of research discussed in later chapters. In the history of edu- 
cation and foreign education, during the earlier years, a large 
proportion of the theses dealt with educational history in the 
United States; during the past few years, with the influx of 
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an increasing number of foreign students, systems of educa- 
tion in other countries are being studied. The International 
Institute of Teachers College, Columbia University, has been 
especially active in this field. 

As a field for research preschool and elementary education 
has not received adequate attention in Doctors’ dissertations, 
although a marked increase in the number of theses in this 
field is noted in 1927, 1928, and 1929. Possibly the fact that 
many elementary-school workers have not felt the urge to 
secure a Doctor’s degree explains the relatively small number 
of studies in this field. Secondary education receives a fair 
proportion of attention. Since 1924, a decided increase in the 
number of dissertations in higher education is noted. Nearly 
one third of the theses under way in 1930-31 dealt with prob- 
lems of higher or adult education. 

The annual bibliography of research, issued by the United 
States Office of Education, includes an interesting tabulation 
of the number of studies in the various subdivisions of edu- 
cation made during the school year covered. The results for 
1929-1934 are found in Table IV.*® 

Three hundred and sixty-eight institutions and organiza- 
tions are represented in the 1929-30 data of Table IV. Of 
4,051 studies listed, about 55 per cent represent Masters’ and 
Doctors’ theses, over 15 per cent periodical references, and 
the remaining entries are for studies of city, state, and various 
independent organizations. For purposes of comparison, the 
122 subjects included in the bibliography were combined and 
reduced in the table to twenty-five main topics in order that 
some idea may be had of the fields which were receiving most 
attention. In later years the number of major headings has 
been reduced to fewer than twenty-five. Table IV shows in 
percentages the relationship between each subdivision and the 
total for the year. Although these percentages indicate to a 
certain extent the subjects in which there is most interest, 


13 Edith A. Wright, Bibliography of Research Studies in Education, 1929-30, p. xi. 
Office of Education Bulletin No. 12. roar. Washington: Office of Edveation. Also 79 
‘ater annual biblfographies. 


TABLE IV 
PERCENTAGE OF RESEARCH STUDIES FOR 1929-34 IN VARIOUS FIELDS OF 


EDUCATION 
Approximate Per Cent 
Subject er 

J 1929-30 | 1930-31 | 1931-32 | 1932-33 | 1933-34 
Educational sociology........ 1.58 
Educational history and bi- 

ography: sap eeraa e TET 1.50 2.38 2.78 3-44 2.51 
Current educational condi- 

tions—United States- ..... 1.50 1.98 1.68 1.14 Ti 
Current educational condi- 

tions—foreign countries, in- 

ternational aspects......... 1.25 2.18 1.60 1.33 1 89 
Educational theory and prac- 

tice, special methods, indi- si 

vidual differences.......... 3-50 2.36 2.27 4.50 4.15 
Educational psychology, child 

study, Cte: i aAA EE 3-50 2.61 1.68 2.00 1.52 
‘Testing and research......... 4.50 5-13 4.57 4.00 5.01 
Special subjects of curriculum.| 23.00 | 24.20 | 25.53 | 38.79% | 38.00% 
Elementary education, in- 

cluding preschool, kinder- 

garten, and primary. .. 1.25 1.22 2.26 1.86 1.36 
Secondary education... | 4.00 4.06 5-03 5.08 4.70 
Teacher training and status. ..| 8.50 8.09 5-85 4.72 5.84 
Higher education 2.75 2.91 4.75 4.00 ange 
School administration 6.25 8.09 7-36 a Sa 
School management 8.25 6.46 5.89 }10.56 2048 
School buildings A ee sce 1.47 1.21 1.06 1.42 
School health and physical 

education RAE EES 4.84 5-31 * if 
Play, social aspects, and child 

welfare 2.00 1.86 2.56 .89* i? 
Rural education. . B BAE Eiri e ti pis s 
Moral and religious education.| 3.50 2.70 2.62 * t 
Guidance? UAA ae eee 2.00 2.73 1.44 as ie 
Vocational, agricultural, 

home economics, commer- 

cial, professional education .| 7.00 7-40 8.57 r ‘a 
Civic and military education. .| .5o 
Education of women.......-. -75 -45 
Education of racial groups. ...| 1.50 I.5I 1.60 1.94 1.83 
Exceptional children. . ... 2.50 2.54 2.36 2.78 3.06 
‘Educational extension and li- 

braries, adult education. .... 2.75 3.03 2.62 2.86 2.60 

‘Totals. taco a ae Sera {£00. 100. 100. 100. 100. 


* The percentage for the curriculum heading is larger than for earlier years because 
wf the apparent inclusion of certain former subdivisions under this heading, for example, 
health and physical education, play, moral and religious education, agricultural educa- 
tion, home economics, and commercial education. 


** Administration and management include the former 


and guidance. 


headings of rural education 
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they do not show the real situation in every instance. For 
example, the number of studies listed under elementary edu- 
cation is very small. However, there are many studies classi- 
fied elsewhere which deal with education at the elementary 
level. Such sections as rural education, special subjects of the 
curriculum, school management, and testing will be found to 
contain many references to studies at the different levels. This 
fact should be kept in mind when examining the table. With 
due consideration to the overlapping which occurs in a list 
of this kind, an examination of the percentages reveals that 
the greatest emphasis during the period covered has been on 
the special subjects of the curriculum. Twenty-three per cent 
of the total number of studies in 1929-30 are credited to 
that topic, and this does not include a number of investiga- 
tions in curriculum making which are listed under school man- 
agement. Of the various subjects of the curriculum, English 
leads in interest, with reading and arithmetic following. Of 
special note is the amount of educational research which has 
been done in the field of training and status of teachers. It 
is second in interest to the topic, “Special Subjects of the Cur- 
riculum,” and it also ranks second in the number of Doctors’ 
theses completed in the field. Although Table IV has its lim- 
itations, it will show not only the special phases of education 
which recently have commanded attention, but also the sub- 
jects which graduate students have been investigating. 

The distributions of research for 1929-34 are quite similar,’ 
as noted in the last five columns of Table IV, especially when 
account is taken of the fact that only graduate theses and 
faculty research are included for 1931-32 and only theses for 
1932-1934. In 1931-32 the most popular fields were those of : (1) 

14 Edith A. Wright, op. cit., pp. x-xi. 

15 Ruth A. Gray, Bibliography of Research Studies in Education, 1931-32. Office of 
Education Bulletin No. 6, 1933. Washington: Office of Education, 1933. Pp. xiv + 282, 
, Bibliography of Research Studies in Education, 1932-33. Office of Education 
Bulletin No. 7, 1934. Washington: Office of Education, 1934. Pp. xiv + 350. 

~ Bibliography of Research Studies in Education, 1933-34. Office of Education 
Bulletin No. 5, 1935. Washington: Office of Education, 1935. Pp. xiv + 328. 
Edith A. Wright and Ruth A. Gray, Bibliography of Research Studies in Education, 


1930-31. Office of Education Bulletin No. 16, 1032. Washington: Office of Education, 
1932. Pp. xviii + 460. 
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special subjects of the curriculum; (2) vocational training 
including such related subjects as agricultural education and 
home economics; (3) school administration; (4) teacher train- 
ing; (5) school management; (6) school health and physical 
education; (7) secondary education and junior colleges; (8) 
higher education; (9) testing and research. Of the special 
subjects of the curriculum, those receiving the most attention 
were English grammar and composition, with a total of eighty- 
two entries; music, seventy-seven; psychology, forty-seven ; 
art, forty-six; modern languages, forty-four; social studies, 
forty-four; history, forty-one; and English literature, thirty- 
eight. In 1932-34 the fields of greatest interest were the cur- 
riculum and school administration and management. 

Other workers in such special fields as the psychology of 
learning, abnormal psychology, psychology in general, sociol- 
ogy, personnel research, and higher education have provided 
helpful guidance with respect to the extent various subdi- 
visions of these areas have been investigated. 

On the basis of an analysis of material on learning found 
in sixty-five periodicals over a period of forty years, Davis 
and Ballard have ranked the topics according to the percentage 
of the studies devoted to each problem. The ranking for 
1920-1929 is as follows: 7° 
Permanence of learning k 
Attitudes, incentives, and motivation. 

Relation of physical and mental ability . 

Mental and ideational learning 

Continuous work, fatigue, narcotics, tod ideuge 
Nature and principles of learning 

Transference and interference in learning 

Drives and confidence . 

Motor learning . 

Attention, interest, and ‘effort. b S 
Reception of stimuli and method of ‘presentation $ 
Reasoning, problem solving, reflective thinking . 


Observation, perception, and report . ~ 
Guidance in learning Sh 


16R. A. Davis and C. R. Ballard, “The Development of Research in Learning,” 
Journal of Educational Psychology, XXIII (March, 1932), 226-35. 
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Goodenough’s analysis of trends in modern psychology is 
based on the Psychological Index since 1894 and on Psycho. 
logical Abstracts from 1927 to 1932. She reports in detail the 
relative emphasis given the topics of:*7 sensation and per» 
ception, educational psycholog; , industrial and personnel 
problems, the nervous system, plant and animal behavior, 
childhood and adolescence, mental tests, statistical methods, 
attention, memory and thought, work and fatigue, sleep and 
dreams, feeling and emotion, maldevelopment and mental de- 
ficiencies, social psychology, and abnormal psychology. Other 
analyses of psychological trends are listed in footnote 17. 

Becker’s tabulation of the space apportioned forty-eight 
topics in sociology covers a thirty-five year period and in- 
volves the large divisions of : 18 personality, family, peoples 
and cultural groups, conflict and accommodation groups, com- 
munities and territorial groups, social institutions, social sci- 
ence and the social Process, social pathology (personal and 


17 Florence L. Goodenough, “Trends in Modern Psychology,” Psychological Bulletin, 
XXXI (February, 1934), 81-97. 

Also see R, A. Davis and S. E. Gould, “Changing Tendencies in General Psychology,” 
Psychological Review, XXXVI (July, 1929), 320-31. An analysis of 110 textbooks in 
feneral psychology published over a thirty-nine year period. 

7. Hunt and Carney Landis, “The Present Status of Abnormal Psychology,” Psy- 
chological Review, XLII (January, 1935), 78-90. An analysis of fifteen textbooks in the 


(January, 1934), 211-4, 

M. Louttit, “Psychological Journals: A Minor Contribution to the History 
of Psychology,” Psychological Review, XXXVIII (September, 1931), 455-60. An 
PA of the subject-, country-, and time-distribution represented in psychological 
journals, 

J B. Maller, “Forty Years of Psychology,” Psychological Bulletin, XXXI (October, 
1934), 533-39. An analysis of American and European publications in psychology, 1894- 
1933. i 
K. L. Smoke, “The Present Status of Social Psychology in America,” Psychological Re- 
view, XLII (November, 1935), 537-43. An analysis of topics treated in the literature 
of this field. 

Helen F. Weeks and Others, “A Comparative Study of Recent Texts in Psychology, 
Educational Psychology, and Principles of Teaching,” Journal of Educational Psychol- 
ogy, XXI (May, 1930), 327-40, 

18 Howard Becker, “Space Apportioned Forty-eight Topics in the American Journal 
of Sociology, 1895-1930,” American Journal of Sociology, XXXVIII (July, 1932), 71-8, 

H. G. Duncan and Winnie L. Duncan, “Research Interests in Sociology,” Sociology and 
Social Research, XIX (May-June, 1935), 442-6. Explores the interests of sociologists as 
recorded in the census of research projects published annually in the American Journal 
of Sociology, The data consist of 1,973 projects between the years 1928-1934, and 
afford an opportunity for determining the general trends in sociological research, 
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social disorganization), methods of investigation, and general 
sociology and methodology of the social sciences. 

An analysis of 571 articles on educational personnel, over the 
period 1920-30, indicates the following topical emphasis in 
terms of percentages: ° educational guidance, 26; counseling, 
13; selective admissions, 12; personality traits, 11; social 
life and extracurricular activities, 9; vocational guidance, 7; 
student government, discipline, and good citizenship, 3; ana 
rating scales, 3. 

As reported in Strang’s study, the relative emphases on 
methods of collecting data, in percentages, are: intelligence 
tests, 48; teachers’ marks, 42; questionnaires, 24; question- 
naires to students, 14; questionnaires to others, 10; rating 
scales, 14; office records, 12; achievement tests, 10; special 
records, 8; tabulation of printed material, 6; character and 
interest tests, 4; tests of emotion, 4; daily schedules, 4; inter- 
views, 4; physical and physiological measures, 4; observation, 
4; and tests of motor ability, 2. 

Although Martin’s canvass of periodical literature on higher 
education involves only 132 articles, it is suggestive of cen- 
ters of interest in this area.2° The number of articles devoted 
to each topic is as follows: student personnel, 30; curricula, 
18; instruction, 17; training of college teachers, 14; adminis- 
trative and supervisory policies, 13; standards and standardiz- 
ing agencies, 13; measurement and recognition of good teach- 
ing, ro; and general and miscellaneous, 17. 

2. Authoritative statements of accomplishment and 
knowledge of completed research. Valuable orientation with 
respect to the status of research in many fields of education 
is provided by stimulating discussions in current periodicals. 
Such evaluative treatments reveal to the worker the foundation 
dn which he builds in carrying forward his own activities in 
the same field. Illustrative analyses may be mentioned briefly. 


49 Ruth Strang, “Trends in Educational Personnel Research,” Personnel Journal, X 
(October, 1931), 179-88. 

20C. W. Martir, ‘Problems of Higher Education as Found in Periodical Literature,” 
Peabody Journal of Education, IX (May, 1932), 372-6. 
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The biennial survey of education, issued by the United 
States Office of Education, provides the best available in- 
formation concerning the present status of education of the 
various types and at the several levels of instruction: admin- 
istration and finance; elementary, secondary, industrial, com- 
mercial, home-making, agricultural, art, music, hygiene, and 
physical education; exceptional children: adult, college, and 
university problems; indigenous peoples, radio, research, li- 
braries, boards and foundations, museums, and legislation. 

In 1931 the American School Board Journal devoted an en- 
tire issue to a summary of forty years of progress in the vari- 
ous phases of educational and business administration: 2* city 
school administration, state education departments, state su- 
perintendency, state policies of financing public schools, 
Department of Superintendence, United States Office of Edu- 
cation, public-school business administration, schcol adminis- 
tration, textbooks and textbook making, school architecture, 
school-room ventilation, public recreation, and school seating. 

Strayer reports twenty-five years of progress in city school 
administration in terms of: application of scientific method, 
professional training of school executives, organization of the 
administrative and supervisory staff, organization of the school 
system, recognition of individual differences, curriculum, per- 
sonnel, technique of supervision, visiting teacher, business 
administration, building program techniques, and record sys- 
tems." However, he maintains that further study is needed on: 
local-Federal relationships, local finance, public relations, indi- 
vidual adjustment, elimination of propaganda, failure and re- 
tardation, and more scientific training of the staff. 

Updegraff’s statement of a decade of Progress in public 


21 For example, G. F. Womrath, “Public-School Business Administration in the Past 
Forty Years,” American School Board Journal, LXXXII (March, 1931), 43-4, 118. 

Also see F, R. Noffsinger, “A Century of Progress in Schoolhouse Construction,” 
American School Board Journal, LXXXIX (December, 1934), 39-40; XC (January, 
1935), 21-2; XC (March, 1935), 37-8; XC (April, 1935), 36; XCII (January, 1936), 
52. 

F. R. Noffsinger, “A Century’s Progress in School Heating and Ventilation,” American 
School Board Journal, XCI (July, September, November, 1935), 38-9, 37-8, 36. 

22 G, D. Strayer, “Progress in City School Administration During the Past Twenty-five 
Years,” School and Society, XXXII (September 20, 1930), 375-8, 
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schools includes improvement in: enrolment, attendance, school 
term, consolidation, transportation, rural high schools, reor- 
ganized high schools, special schools and classes, night schools, 
buildings, salaries, instructional procedures, curriculum, and 
Federal relations to education.** 

Workers in educational psychology have provided overviews 
of accomplishments in several subdivisions. Gates discusses 
advances under the following captions: educational objectives, 
teaching and learning, results of present educational systems, 
educational diagnosis and measurement, educational guidance 
and prognosis, educational diagnosis and remedial work, trends 
in methods of research, trends in systematic accounts of edu- 
cational psychology, and use of the school as a laboratory.** 
He also shows the contribution of psychology to education in 
the introduction of scientific method, in providing a view of 
human nature fundamental to modern philosophy in the de- 
velopment ‘and application of a body of scientific principles 
to educational theory and practice, and in providing useful 
professional techniques and instruments.* 

In naming the contributions of science to education, Free- 
man includes increased knowledge of : learning procedures with 
special reference to the school subjects, nature and develop- 
ment of the child, mental tests and individual differences, 
determination of curriculum content, measurement by ob- 
jective achievement tests, and the school personnel. However, 
he suggests that too many studies have been trivial, negative 
in their conclusions, and intended for appraisal rather than 
discovery. 


23 Harlan Updegraff, “Progress in American Public Schools, 1920-30,” Nation’s 
Schools, XIV (August, 1934), 17-20. 

24A. I. Gates, “Recent Advances in Educational Psychology,” School and Society, 
XXIX (January 5, 1929), 1-8. 

25A. I. Gates, “Contributions of Psychological Research to Education,” School and 
Society, XXXI (April 12, 1930), 485-93. 5 

26 F. N. Freeman, “The Contribution of Science to Education,” School and Society, 
XXX (July 27, 1929), 107-12. 3 A 

Also see F. N. Freeman, “The Contribution of Science to the Principles and Methods 
of Teaching,” Childhood Education, VI (April, 1930), 348-54. 

Harold Rugg, “After Three Decades of Scientific Method in Education,” Teachers 
College Record, XXXVI (November, 1934), 111-22. 
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A statement of the contribution of research to curriculum 
construction in the secondary school recognizes the value of 
the reports of committees and survey commissions, but specifies 
difficulties or problems yet to be solved: the ultimate good 
by which to measure all procedures, the gap between cur- 
riculum and conduct, specific causal factors affecting results 
in a given field of instruction, and restraining influences of 
college-entrance and other formal requirements." 

Research in mental hygiene has contributed to: techniques 
for measuring individual adjustment, adjustment of the schoo! 
organization and program to the individual, organized guid- 
ance programs, clinical methods of studying and aiding the 
problem child, and home contacts through the visiting 
teacher.”* 

Probably the foregoing selected statements of the achieve- 
ments of research in given areas will show the type of orien- 
tation provided in the selection of a problem for investigation. 
However, for the sake of completeness and to provide a vari- 
ety of such statements of progress, other similar references 
may be found in the supplementary bibliography at the end 
of this chapter. These analyses include: 


A century of teacher training . . . . French 

A century of progress in education . . Judd 

A century of progress in psychology . . Flugel 

A century of progress in social science . Bernard < 
Junfor-college). 0) Sie). a Fda ey eta eelis Noffsinger 
Organization of secondary schools . . . Wiley 
Secondary curriculum . . . . . . Loomis 

Method in secondary schools . . . . Douglass 
Education for girls . . . . . . . Wilson 

Study (isbn), eee Os i shoes et dy (Woodring-and Fleming 
Principles of teaching . . . . . . Freeman 
Educational sociology . . . . . . Smith 

Science teaching . . . . . . . . Caldwell, Curtis 
English TN Seco Geen awe ayers: ey GT ewett, 


27H. R. Douglass, “The Contribution of Research to Secondary School Curriculum 
Construction,” School and Society, XXXII (September 27, 1930), 411-6. 

28 P. M. Symonds, “The Contribution of Research to the Mental Hygiene Program 
for Schools,” School and Society, XXXIV (July 11, 1931), 39-49. 
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Student personnel . . . . . . . Strang 
Measurement of character . . . . . Shuttleworth 
Mental hygiene . . . . . . . . Ruggles, White 
Parent education . . . . . . +. ~~ Bridgman 


Additional assistance in canvassing completed research may 
be found in Chapter III which is a detailed treatment of the 
sources of information available in making an exhaustive bib- 
liographical survey of investigational activity in a given field. 
Obviously, the educational worker does not wish to attack a 
problem which has been satisfactorily solved. On more than 
one occasion a research worker has spent much time in the 
investigation of a problem only to find that some other person 
has previously arrived at the same conclusion. In some cases 
the results were even available in print, but the investigator 
in question had failed to make a thorough bibliographical 
survey. Probably graduate students are more likely to dupli- 
cate the work of some student in another institution, due to 
the difficulty involved in ascertaining the problems being in- 
vestigated by students in graduate schools. Examples of such 
duplication are given in Chapter III. Familiarity with com- 
pleted research reveals gaps in the available knowledge of a 
given field, and suggests studies which need further investiga- 
tion or repetition for purposes of verification. 

3. Analyses of educational trends. Critical insight with 
respect to current problems and familiarity with changes in 
education are promoted through examination of articles deal- 
ing with trends in given fields. For the benefit of their readers, 
who are classroom teachers in many instances, most educa- 
tional journals feel obligated to prepare occasional analyses 
of such changes in the areas represented. A method employed 
rather frequently in discovering shifts of emphasis is to 
analyze journals over a period of years, as described earlier 
in this chapter.2? Twenty-three addresses delivered at the 


29 Howard Becker, “Distribution of Space in the American Journal of Sociology, 1895+ 
1927,” American Journal of Sociology, XXXVI (November, 1930), 461-6. 

Mark Fawcett, “A Comparison of the Educational Problems of 1910 and 1926, as Re- 
vealed by Educational Magazines.” Unpublished Master’s thesis, University of Chicago, 
1928, 
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twenty-fifth anniversary of the College of Education, Uni- 
versity of Minnesota, picture changes occurring during the 
twentieth century.” Other illustrative analyses of educational 
trends may be found in the chapter bibliography and deal 
with: 


Teaching of mathematics . . . . . .... Year-book 
Characters education: seasea c esi en ner Wiley 
Personnel research < T E ey eee ee eee Strang 
Education in the United States . . . . on. Judd 
Recent social changes in relation to educational research Willey 
Educational sociology . . . ....~, eee ay re 
Junior-college curriculum . . . . . , =. Barton 
Curricula Seca emcee ote ead MAC) Se ON, Zirbes 
Mea sarernen CRMs MIRAI HD eae TS Oh eta ss sl Segel 
by stealmecticabontiay, ss lee T a A L aie E osvure Cassidy 
Business English SCPE Met Sie ON Bole oe r Aner 
Modern foreign languages. . . , + e . . Henmon 
Guidance Coe as! seam ee cee ae ee ee ye) IP TOCtOn 
Mental hygiene. 28 on AEL - . + Ruggles 
Secondary scchoolsis els sks ieewes cet dul oc . Hotz 


4. Illustrative critiques of given fields. Occasional discus- 
sions of investigational results Secured, techniques employed, 
or shortcomings in certain areas of activity reveal gaps in the 
available information which should be closed up by the results 
of further study. Such critiques are represented by a diversified 
group of references in the chapter bibliography on: 


Educational research . . . . . Barr, Coxe, Davis, Lyon, Rugg 
Examinations) ea aa «sn. AEM CCORA 

Effects of measurement of instruction Woody 

Measurement in the secondary school Odell, Stephens 


Personality measurement. . . , Hendrickson 

Secondary education . . . . . Douglass, Edmonson, Everett, 
Reavis 

Research in higher education, . . Henmon 

Teacher preparation . . , .+ + Barr and Douglass, Myers 

Progressive education or the activity 

movement . . . e . e . Dickson, Freeman, Hissong, 

Tyler 


30 The Changing Educational World, 1905-30. Edited by Alvin Eurich, Minneapolis, 
Minn.: University of Minnesota Press. 1930. Pp. 312. 
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Guidance . . . . . . . ~ Boynton, Kefauver 

Philosophy of education . . . . Hiullfish 

Ability grouping . . . . . . Otis 

Science teaching . . . . . . Curtis, Riedel 

Curriculum . . . . so . - « Judd, Everett 

Courses of study . . . . . - Harap 

Supervision. . . . . a « ~ Gambrill, Goodykoontz, Ran- 
kin 

Methods of teaching . . . . . Thorndike 

Arithmetic and reading . . . . Brownell 

GTEGEEVES#n ?. > 5m) & oven! siinisysliceneeler 

Educational reconstruction . . . Wheeler 


5. Educational prophecies. Current literature contains in- 
teresting predictions of future developments in education. Ex- 
amination of such prophecies offers suggestions concerning 
research which must be accomplished before the predictions 
made can be realized. 

In 1931 members of the Department of Education, Uni- 
versity of Chicago, placed in the corner stone of their new 
building prophecies of the status of education in 2031. A sum- 
mary statement of the prediction follows: ** 


Careful comparative reading of the prophecies discloses considerable 
recurrence of similar elements. One of the predictions most frequently 
emerging pertains to the status of the science of education. One 
writer goes so far as to anticipate that in a hundred years scientific 
knowledge regarding education will supplant entirely appeal to per- 
sonal authority. This type of prediction is often associated with one 
relating to concurrent development of a science of society or of human 
nature. Other elements of frequent recurrence refer to the extension 
of opportunities of universal education, adult education, the curric- 
ulum, individualization, and school organization. Popularization of 
education will be carried to higher age levels, and the recent move- 
ment for continuous education through adulthood will have moved 
toward universalization. The curriculum will have been modified in 
a number of ways, including more attention to cultural and non-voca- 
tional phases on levels below the professions. The organization of 
schools promises to include three levels: an elementary period (certain 
prophecies also mentioning nursery-school developments), a secondary 
period that will extend to the end of general education and displace 


` 81 Members of the Department of Education, “Education in the Year 2031,” School 
Review, XXXIX (May, 1931), 337-49- 
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the four-year college, and the university proper. Among elements re- 
currently but less frequently mentioned than those listed are improved 
procedures in teaching (refined through experimentation), programs of 
guidance, and improved articulation of the system of schools. A num- 
ber of predictions are mentioned once or twice only, among them 
several of a non-educational character, such as progress toward a 
universal language, the disappearance of common labor, and the dis- 
placement of the state as a governmental unit. 


Briggs first enumerates facts, both favorable and unfavor- 
able, out of which prophecy might grow and vision blossom, 
and later makes predictions concerning the secondary school 
of the future. The facts of the present are: *? 


1, Lack of fundamental thinking that has materially affected sec- 
undary education 
2. A stifling satisfaction with tradition 
3. The transcendent faith of the American people in education, 
which can be used to Support a sound educational program 
4. A machinery of education unparalleled elsewhere in the world, 
which should be put to better use 
5. A too thoroughly standardized system of secondary schools 
6. Need for further improvement in some respects in materials of 
instruction 
7. The challenge to, and opportunity of, democracy by reason of 
the vast numbers of pupils in secondary schools 
8. Individual differences not adequately provided for by a suff- 
ciently diversified program 
9. New psychology and new philosophy not yet consistently a 
part of educational practice 
10. Lack of sufficient confidence on the part of teachers in the 
subjects taught 
11, The small accomplishment of the majority of pupils 
12. Need to recruit and train a more competent body of teachers 


His vision of secondary education in the future includes 
the following concepts: general recognition by the public that 
secondary schools are vitally important ; realization that ef- 
fective education cannot be divorced from large social aims; 
a much more comprehensive school; no contest between the 
materially utilitarian and the cultural education; an extensive 


82T. H. Briggs, “A Vision of Secondary Education,” Teachers College Record, XXXIV 
(October, 1932), 1-37. 
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program of comprehensive planning; education for intelligent 
participation in a democracy; equal opportunity for each to 
advance his talents for the good of society; teachers guaran- 
teed permanent tenure; and a more consistent demand by 
society for the product of the schools. 

Long predicts that the major emphases of secondary edu- 
cation during the next decade will be: (1) attack on the prob- 
lem of economic life, (2) increased capacity for consumption 
of the real values of life, (3) suitable instruction for all pupils 
of secondary-school age, (4) training to meet the needs of a 
rapidly shifting vocational world, and (5) a better under. 
standing of peace and of the futility of war. Other adjust- 
ments of the future predicted by Long are: breakdown of the 
time-to-be-served theory, independent study, realignment of 
departmental barriers, removal of the tyranny of college domi- 
nation, better use of radio and films for educational purposes, 
apprenticeship training of teachers selected on the basis of per- 
sonality and creative ability, extension of the high-school li- 
brary service especially during vacation periods, gradual reduc- 
tion of the social distance between teacher and pupil, and 
incorporation of the junior college as a part of the public. 
school system.** 

Another forecast of changes in the secondary school fol- 
lows: * 


1. The college preparatory-academic domination of the curriculum 
will be discontinued, and the curriculum will be determined on the 
basis of life values. 

2. The conventional departmentalization of the curriculum and 
the organization of courses according to academic patterns will be 
displaced by a departmentalization based on broad fields of experience. 

3. Curricular experiences of students will be sequential; specialized 
semester and yearly courses will be displaced by a continuous program. 

4. Students will participate in the planning of the school program. 

s. Individual differences will be given increasing attention not 


83 F, E, Long, “Secondary Education During the Next Decade,” Junior-Senior High 
School Clearing House, VIL (September, 1932), 11-7. 

34W, L. Wrinkle, “American Secondary Education and the Future,” Junior-Senior 
High School Clearing House, X (December, 1935), 239-46. 
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by the multiplication of specialized courses but by the recognition of 
differences in interests, needs, and abilities within general fields of 
experience. 

6. The conventional grade classification and annual promotion of 
students will be discarded. 

7. The use of the Carnegie unit and similar quantitative measures 
will be abandoned, 

8. Ccmpetitive marking systems and school marks will be elimi- 
„nated; failure will become impossible. 


Frederick discerns thirty-six trends in the secondary school 
of the future: ** 


1. In the high school of the future will be found 90 per cent 
of children of high-school age unless the C.C.C. idea or the militarists 
beat us to the youth of the land. 

2. The six-year secondary school will become more frequent in 
smaller communities. 

3. The senior high school will become more like the present-day 
junior high school, at least in form. (Guidance and extracurricular 
activities are now listed as constants for the senior as well as the 
junior high school by the State of New York authorities.) 

4. The period of secondary education will be extended by some 
form of Post-graduate organization or junior college, 

5. The high school of the future will tend more and more to 
become a community high school in the sense that it will take a more 
definitive part in, and draw more extensively on, the life of the com- 
munity by which it is supported. 

6. Support for the system of secondary education will more and 
more be drawn from the state treasury, 

7. Small high schools of fewer than one hundred pupils will dis- 
appear as transportation facilities improve and the population moves 
from marginal lands. 

8. Cities will continue to build cosmopolitan high schools with 
only occasional abortive attempts to segregate pupils into particular 
trade schools. 

9. In cities the typical secondary-school organization will show a 
three-year junior high school, a three-year high school, and a two-year 
junior college. In some instances, of course, a 3-5 or an 8 or a 6-2 
plan will be set up, but the 3-3-2 plan will prevail. 

to, Pupils of the secondary school of the future will have a larger 
say in the conduct of the school, They will help select their books for 


35.R, W, Frederick, “Future Trends of the High School,” Curriculum Journal, VI 
(May. 22, 1935), 16-9. 
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general reading, topics for social studies, and of course their social and 
extraclass activities. 

11. The high-school principalship will follow the present trend and 
become eventually an office of high professional status requiring 
thorough and specific training. I look for the principal to become one 
of the four most important persons in all communities except the very 
largest cities. 

12. The training of the high-school teacher of the future will be 
dominantly social and psychological rather than academic. Professional 
training will take precedence over conventional majors and minors. 
[This is a sharp break from past practice.] 

13. The teacher for the high school of the future will be trained 
for and hired because of ability to carry on general child guidance or 
advisory functions, rather than for academic mastery of major and 
minor subjects. High-school teachers will eventually become teachers 
of boys and girls rather than of subjects. 

14. Extracurricular or “related activities” will continue to grow in 
respectability. This does not mean that definite credit will be given, 
for our notion of credits will be very much modified. 

15. Individualization will be broadened to insure attention to dif. 
ferences in social-economic status, emotional stability, and nervous 
differences. Ability grouping on the basis of intelligence will be aban- 
doned as an unnecessary and monstrous violation of the dignity of 
youth. 

16. Marks, report cards, failure, and retardation will be elimi- 
nated in the future high school. Every child who wants one, will be 
able to get a diploma in the same number of years. 

17. High-school diplomas of the future will be more specific in 
their presentation of the accomplishment of pupils. All pupils will get 
the same size diploma with the same size bow of ribbon. The account 
on the inside will tell the story to all who may be interested. 

18. The business in educational tests for science, history, Latin, 
etc., will collapse, for secondary-school teachers of the future will see 
their lack of validity for the social objectives of education. 

19. Vocational guidance will become only a minor phase of guid- 
ance activities. The field of guidance will in the future concentrate on 
the more important factors of personality, character, happiness. 

20. Vocational education as a terminal program will be gradually 
shifted to the years beyond the twelfth grade. The free public high 
school will stop preparing stenographers, printers, and milliners. 

21. There is a trend toward the elimination of textbooks to be 
learned. Instructional materials of the future will be more flexible, 
more in the form of guides to classroom activity, and they will be 
more and more often prepared by the local staff. 

22. The high school of the future will capitalize all learning situa- 
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tions in the life of the school including corridors, cafeteria, playground, 
and discipline. 

23. More and more of the secondary-school curriculum will move 
into the required, constant variety. There will be less time for free 
elections. The program will be of the constants-variable type, with few 
variables. $ 

24. The content of the curriculum of the future high school should 
be drawn from the general areas of human interest and need as health, 
leisure, religion, philosophy, rather than from the traditional hierarchy 
of subject disciplines as mathematics, sciences, history. 

25. Secondary education of the future will see the gradual elimina- 
tion of dividing lines between subjects. Science and mathematics will 
come together, and history, economics, geography, and civics will com- 
bine into a general course in citizenship. In fact the day may come 
when pupils will study for a whole year one topic, in which all fields 
are combined. 

26. The high school of the future will, let us hope, give much more 
time for all students to the fine arts and music. 

27. The history sequence will disappear. In its place will be offered 
a study of contemporary civilization in which students will incidentally 
dip into the past experiences of the race. 

28. Group prescriptions as for college entrance or commerce groups 
will be very much less frequent and distinctive in the future. Common 
integrating knowledges and skills will be the point of emphasis. 

29. The forward reference will be minimized in the secondary 
school of the future. Each year’s work will be planned and conducted 
for its own sake. Pupils do not go through the ninth year so they can 
do the tenth year’s work. 

30. The secondary school of the future will attempt to arouse in 
each child an interest in the welfare of mankind on a planetary scale. 

31. The secondary school of the future will avoid deliberate indoc- 
trination and strive to eliminate unconscious imposition of patterns of 
thinking. 

32. The high school will continue to prepare for college, but the 
present trend will continue until finally detailed specifications of 
particular subjects will give way to general character and personality 
development and mastery of study-learning skill. 

33. Secondary education will eventually cease to be an institution 
devoted primarily to the imparting of academic information. The 
school will gradually come to emphasize the immediately practical 
habits and skills necessary for the daily job of living. 

34. The secondary school of the future will give far more attention 
than at present to the subtler phases of child development including 
interests, special talents, and personality. 

35. Primary attention in the future will be directed to the task of 
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making pupils adjustable to a changing society rather than to the 
problem of adjustment to any one social and economic system. In this 
direction lies the hope of preserving democracy. 

36. Secondary education of the future will be bolder in that teach- 
ers and pupils will be trusted to consider all sides of controversial 
issues without the imposition of any viewpoint by the teacher except 
that facts must be respected. 


Snedden ** speaks in the tense of 1960 and describes Ameri- 
can high schools and vocational schools as existing then in a 
society no less complex than that of 1931 but with larger social 
controls to utilize scientific techniques, govern production, 
diffuse prosperity, and to attain the elements of human welfare 
for all. With home and community life made richer and there- 
fore more soundly educative for the young, very few Americans 
will send their children under nine years of age to school. 
Attendance on full-time schools of general (interpreted as 
strictly non-vocational) education will be compulsory through- 
out the United States from the tenth to the eighteenth birth- 
day for all. From eighteen to twenty-five full-time attendance 
on some type of vocational school will be required of all during 
a specified number of weeks, and a comprehensive scheme of 
such schools is outlined by Snedden. There is complete 
elimination of the vocational aim and the college-preparatory 
function from high-school education. To the realization of 
“really functional cultural and civistic attainments” the pro- 
gram of the high school is directed. 

Certain anticipations of the development of American col- 
leges and universities by 1980, expressed by the same author, 
include the following concepts: a greatly increased number 
of distinctively professional colleges within the larger uni- 
versities, evolution of graduate studies into distinctly pro- 
fessionalized curricula, culmination of interdependency of lib- 
eral and professional colleges, and the freeing of the liberal 
college to offer courses only of proved functional significance 


86 David Snedden, American High Schools and Vocational Schools in 1960. New York: 
Bureau of Publications, Teachers College, Columbia University, 1931. Pp. vi +- 122. 
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for civic competence, personal culture, and other non-voca- 
tional achievement.** 


Hughes believes that research in public education by 1950 
may be characterized by the following statements: ** 


I. Shift from minor to major problems, from the narrow field of 
tests and measurements to every important phase of educational 
activity 

2. Having determined codperatively educational objectives, pro- 
cedures, and facilities, research with the continuous purpose of evaluat- 
ing them in terms of consistency and effectiveness 

3. Revelation of the incompatibility of older school standards and 
of life standards in general 

4. Facts and principles of efficient personnel administration deter- 
mined, finding much in common with personnel management in busi- 
ness and industry 

5. Evaluation of many new departures in education 

6, Revelation of discrepancies between real and supposed educa- 
tional practice 

7. Evaluation of actual school procedures 

8. Determination of causes of inarticulation between the school 
and the home 

9. Discovery and analysis of the qualities of personality essential 
for success in the various occupations and professions 

10. Evaluation of the 6—4—4 plan, of secondary school organiza- 
tion 


A prediction of the status of education in 1940 ends with 
the following conclusion: *° 


It is predicted above that the enrolments in all levels of Education 
in 1940 will be as follows: Kindergarten and Elementary, 26,000,000; 
Secondary, 7,000,000; Normal Schools and Teachers’ Colleges, 175,- 
000; Colleges, Universities, and Professional Schools, 1,250,000; and 
Special Schools, 700,000. It is estimated that the teaching of the 36,- 
00,000 students of 1040 will be carried on by 1,200,000 teachers. 
Soon after 1940, all elementary teachers will have four years of col- 
lege training, while secondary teachers will have five years, and junior 
college teachers will have six years of college training. Moreover pro- 


87 David Snedden, “Some Anticipations: American Colleges and Universities in 1980,” 
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39 F, C. Touton, “Education in 1940,” Education, LIII (March, 1933), 428-35- 
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gressive college teachers will have had training in educational methods. 
Teaching in 1940 will consist in directing student experiences toward 
clearly defined objectives with appropriate content. Classrooms will 
be built and equipped with special reference to subjects taught within 
their walls. Methods of learning will consist of a conscious well- 
directed attempt to develop abilities of individual students; films will 
be used to present scenes in literature and history, and processes in 
physical and biological sciences. Students will be trained in organizing 
materials and making judgments in classroom activities, A clearer 
vision of the need for education in our democracy will lead in the 
direction of state-supported education and equalized educational oppor- 
tunities through the secondary-school period. 


In the chapter bibliography are represented other interest- 
ing attempts to look into the future of a variety of educa- 
tional areas: 


A survey of a century of British education and 


a forecast. eo ge te e bi tia este dias ORODLIN 
Twenty-five years of prophetic achievement . Morgan 
Development of public education . . . . Norton 
The outlook in secondary education. . . . Sadler 
The outlook for school research . . . . . Osburn, Theisen 
The future of school administration. . . . Chambers, Osburn 
Next steps in curriculum making . . . . . Bode, Briscol, Da- 

vidson, Harap 
Future progress in supervision. . . . . . Burton 
Prospects in home-economics education . . . Harap, Hopkins 
The program of studies . . . . . . . Loomis 
Mental hygiene’ vel hee pte etre we 0-0 ee Enek 
Mathematics training . . . . . . - . Jablonower, Marsh 
Teacher education... «© . «. + «© - .« Cooper 


6. Research under way. The type of information found in 
Chapter III provides assistance in determining studies in 
progress. Reference also should be made to another helpful 
guide for accomplishing the same purpose.‘® Of course, it is 
undesirable in most cases to duplicate an investigation either 
completed satisfactorily or well under way. The material of 
Chapter III may be anticipated at this time by certain rather 
specific suggestions for canvassing research in progress. 


40 Carter Alexander, How to Locate Educational Information and Data, pp. 223-9. 
New York: Bureau of Publications, Teachers College, Columbia University, 1935. 
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Through their administrative offices, certain graduate schools 
make available a list of current investigations on which stu- 
dents and staff are engaged. Certain journals are attempting 
to make information available concerning graduate or staff 
studies in progress in the fields of education * and sociology,‘ 
for example. News items in periodicals provide valuable in- 
formation concerning future year-book topics, school surveys, 
projects of research agencies, and other important investi- 
gations under way. The Journal of Educational Research 
through its Department of Research News and the School Re- 
view and Elementary School Journal through their editorial 
columns have been active in providing such information. Ad- 
vance notices of publications issued by teacher-training in- 
stitutions, research bureaus, and educational publishers are 
worth reading to maintain contact with studies well under way 
or near completion. From time to time news-letters of chap- 
ters of Phi Delta Kappa (national educational fraternity) have 
printed lists of the projects on which members were engaged. 
Usually the year-books of professional organizations and 
learned societies include a list of the topics for projected year- 
books on which work is going forward. The Office of Educa- 
tion at Washington has also provided assistance. As early as 
February, 1926, and in March, 1927, the National Committee 
on Research in Secondary Education published through the 
United States Office of Education a Bibliography of Current 
Research Undertakings in Secondary Education. In March, 
1928, the Office published a Bibliography of Current Research 
Studies in Education. This series of reports has been con- 
tinued in one form or another.** The annual report of the 


41 Carter V. Good, ‘Doctors’ Theses Under Way in Education, 1934-35,” Journal of 
Educational Research, XXVIII (January, 1935), 376-09. This annual service began in 
1931. 


» “Doctors’ Theses, Under Way in Education, 1935-36," Journal of Educational 
Research, XXIX (January, 1936), 384-406. 

42 “Students’ Dissertations in Sociology,” American Journal of Sociology, XL (July, 
1934), 74-103; XLI (July, 1935), 67-100. 

G. A. Lundberg, “Current Research Projects (Sociology),” American Journal of Sociol- 
ogy, XL (September, 1934), 221-39; XLI (September, 1935), 218-35. 

43 Edith A. Wright, List of Educational Research Studies of State Departments of 
Education and State Education Associations, 1931-32. Office of Education Circular No, 
63, October, 1932. Washington: Office of Education. Pp. 44. 
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Commissioner includes a list of the projects under way at the 
Office of Education. The annual reports of foundations and 
research organizations usually enumerate lists of studies in 
progress. 

In this connection, a discussion of sources of information 
covering research in progress in the humanistic and social 
sciences is of interest.** Certain other specific instances may 
be cited to illustrate the way in which state, city, and uni- 
versity organizations make known to the public the studies 
which are being carried forward by the agency or its members: 
a university research bureau (Ohio State),*° California studies 
in secondary schools, Sacramento projects,“ public-school 
research in Michigan,** collegiate research at Minnesota,*® 
research in New York," North Carolina studies in progress, 
and a description of investigations promoted by research or- 
ganizations in Ohio.** 

It is recognized that when these paragraphs reach the reader, 
the projects listed as under way at this writing will no longer 
be of contemporary interest. However, many of the sources 
of information mentioned are more or less continuous, and the 
interested worker has only to secure later numbers of a given 
bulletin, report, or periodical to become acquainted with cur- 


44 Lawrence Heyl, “Sources of Information Covering Research in Progress and Uni- 
versity Dissertations,” School and Society, XXIX (June 22, 1929), 808-10. 

45 ‘The Bureau of Educational Research in 1932-33 (Ohio State University),’” Edu- 
cational Research Bulletin, XII (November 15, 1933), 227-38. 

46F. C. Touton, “Research Projects of the Secondary School Level Carried on in 
California Cities During 1932-33,” California Quarterly of Secondary Education, IX 
(January, 1934), 167-94. 

47 J. F. Bursch, “Research Projects in the Sacramento City Schools,” California 
Quarterly of Secondary Education, V (January, 1931), 186-8. 

48 Clifford Woody, Summaries of Investigation Reported by Michigan Directors of 
Educational Research. Bureau of Educational Reference and Research Bulletin, No. 130, 
Ann Arbor, Mich.: University of Michigan, 1930. Pp. 48. 

49M. E. Haggerty, “Institutional Resources Available for Collegiate Educational Re- 
search,” School and Society, XXIX (May 25, 1929), 653-64. 

50 “The Study of Secondary-School Problems in New York State,” School Review, 
XXXVIII (June, 1930), 40-4. 

“Research Projects of the Bureau of Reference and Research of the New York City 
School System,” Elementary School Journal, XXX (October, 1929), 94-5. 

51M. C. S. Noble, Current Reports on Activity Programs, Projects, Research Studies, 
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intendent of Public Instruction, 1929. Pp. 48. 
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Record, (October, 1930), 1-107. 
52 “Mapping Research Areas," Ohio Schools, VIII (June, 1930), 248. 
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rent investigations of succeeding years. At least, the references 
cited illustrate the type of information concerning research in 
progress now being made available. Other illustrations of 
available information on current research deal with student 
teaching, contemplated year-books on supervision to be pre- 
pared by the Department of Supervisors and Directors of 
Instruction,*™* kindergarten research,” work and projects of 
the National Council on Education," personnel studies,®* and 
investigations in Great Britain.°* 

7. Needed research in education. The educational literature 
of the past ten years contains numerous analyses of problem 
areas in need of investigation. Although many field workers 
are conscious of their own problems, there is a tendency to 
overlook needed research because of too close contact with 
routine duties and specific details of the given situation; that 
is, “the woods can’t be seen because of the trees.” It is said 
of those who live near the Alps that they rarely cast up their 
eyes to appreciate the grandeur which thousands journey 
around the world to see. Therefore, a list of needed studies, 
compiled by an educational worker with vision and insight, 
and possibly detachment, frequently proves stimulating to the 
individual in close contact with the field represented, as well 
as to the more remote student of the same problems. The 
foregoing statement is made with full recognition of the fact 


53A. R. Mead, “Report of the Research Committee, the Supervisors of Student- 
Teaching,” Educational Administration and Supervision, XV (May, 1929), 385-8. 

A. R. Mead, “Studies of Training School Problems,” Educational Administration and 
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55 “International Kindergarten Union Research Committee: Report of Progress,” 
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57 W. V. Bingham, “The Personnel Research Federation in 1930,” Personnel Journal, 
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that such field workers as administrators and supervisors have 
provided penetrating analyses of needed research in their areas 
of responsibility. It is also recognized that the graduate stu- 
dent with little or no field experience frequently has difficulty 
in isolating problems because he is too far removed from 
concrete school situations. 

In some instances graduate schools and departments of 
education, individual professors and directors of research, and 
occasionally field supervisors have compiled lists of problems 
for investigation, to be placed in the hands of the graduate 
student or field worker. This service may possess certain 
values for the beginner in research, but too frequently such 
lists of problems are considered as “assignments,” with de- 
mands made on the person who compiled the list to suggest a 
plan and technique for the solution. Ordinarily the worker 
who chooses his own problems for study, of course using, to 
the extent desired, types of assistance enumerated in this 
chapter, thereby indicates a degree of intellectual inde- 
pendence and promise not possessed by the individual who 
asks for a research “assignment.” The alert worker should 
find many stimulating suggestions for problem solving in con- 
nection with the pursuit of university courses of instruction 
and other types of reading and study. In fact, many careful 
reports of completed research end with a statement of leads 
for further investigation. 

In the appendix are suggested lists of problems requiring 
investigation in a number of the subdivisions of education. 
Two factors of significance are the recent publication date of 
many of the analyses and the wide range of interests repre- 
sented. The critical, inquiring attitude represented should prove 
stimulating to teacher, graduate student, and other educational 
workers. It is recognized that some of the problems enumer- 
ated will have been at least partially solved by the time this 
chapter is in the hands of the reader. However, the biblio- 
graphical survey described in the next chapter will reveal to 
the investigator those issues which are still current. A number 
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of the digests of research and bibliographies cited in Chapter 
III suggest problems for investigation. Owing to limitations of 
space and the varied interests of the readers it does not seem 
desirable to reproduce in the appendix complete lists of the 
issues recommended for investigation. In some instances, only 
major divisions into which the problems fall are given. Having 
made the selection in terms of his own area of specialization 
or interest, the educational worker may read profitably the 
discussion of needed research in its original setting. However, 
sufficient description will be provided to identify the type of 
problem discussed by the various authorities cited. 

In no sense should the brief analyses in the appendix be 
considered as specific statements of definitely delimited prob- 
lems for investigation, but as guides for future study and as 
raw materials for formulating problems. No attempt has been 
made by the present authors to determine critically the most 
urgent research needs, except through selection of the writings 
summarized. Welcome as a single comprehensive list of needed 
investigations in the entire field of education, with relative 
values indicated, might be, it is doubtful whether any one or 
even any three workers have sufficient insight and expertness 
to perform such a service. Possibly identification of needed 
research in any important subdivision of education is a suf- 
ficiently large task for a single educational worker. The com- 
bined analyses of many such experts in different areas should 
present a reasonably complete picture of needed investigations 
in the entire field of education. This, in effect, is what is pre- 
sented in the appendix, although undoubtedly some areas are 
analyzed more fully than others, and certain smaller subdi- 
visions have been treated briefly or even omitted. 

The plan of classification employed involves a limited num- 
ber of major subdivisions of education, with an alphabetical 
arrangement by authors under each large heading. Such a 
scheme seems more usable for present purposes, particularly 
in seeking to determine quickly whether or how a given leader 
has analyzed the needed research in his area of specialization 
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and competence, than to attempt an elaborate system of sub- 
topics under the major headings used. As a matter of fact, 
the number of references under a given heading is hardly large 
enough to make further sub-classification by topics necessary 
or even desirable. 

The fields represented are: (a) higher education, (b) sec- 
ondary education, (c) elementary education, (d) curriculum, 
(e) school subjects, (f) educational psychology and measure- 
ment, (g) supervision, (4) administration, (¢) rural education, 
and (j) general problems. Certain of the analyses could be 
classified under more than one heading, for example, a dis- 
cussion of needed research in secondary-school measurement 
or in college French, Therefore, it will be necessary in the 
case of certain areas of interest to canvass more than one sub- 
division. The fact that in some instances the same problem is 
emphasized by more than one author should not be viewed as 
useless repetition in these analyses, but as a partial indication 
of the urgency with which a given area demands study. 

In the appendix a few of the statements of needed research 
are in the form of a connected discussion or exposition rather 
than a numbered list of problems. The reader who prefers this 
type of presentation may usually find it in the original source 
from which a given list of problems was taken. 

In addition to the authoritative statements of needed inves- 
tigations, reported at some length in the appendix, a number 
of other analyses can be mentioned only by the topic repre- 
sented and, in most cases, by the name of the author, as listed 
in the chapter bibliography: 


Miscellaneous educational problems . Judd, McCall 


Educational reconstruction . . . . Wheeler 

The summer camp. . . . . ~- McClusky 

Prevention of delinquency . . . . Olson 

Comparative education. . . . . Eckelberry 

Supervision ci mias yaoi epee tae ee 

Administration . . . . . . «e Ashbaugh, Ackerly. Wiley 
College entrance. . . . . . . Toops 


Teacher training . . . . . . Hughes 
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Secondary education . . . - . Douglass, OBrien, Sadler 
Girriculum a uan Kars =p aoe eude 
Progressive education e. = < o Freeman 
Experimental possibilities in public 
achoa re aW a y gust Ree Meyers 
Visual adel e saorta NDE, 
Religious education . . . . . . Watson 
Art education | syn) a sms ete aa WATO 
Industrial education. . . . . . Smith 
Home economics . . . . . + Kyrk 
Foreign languages . . . . . . Cole, Hutchinson 
GeOgTADDY. ns54.-s ye 0s oso 0) ae -Atwood 
Physical education . . . . . . Browne, Griffith 
Child study . . . . . . . . Goodenough and Anderson 
Elementary education . . . . . Cole, Goodykoontz, Klapper 
Measurement mutase Eels yh slepise j teas EROLDY: 
Mental and physical growth . . Brooks, Meredith and Stod- 
dard 
Mouths ea) ia. Ma) Sheree alee Elubbard 


SELECTING, STATING, AND DEFINING THE PROBLEM 


Selecting a problem for investigation. If more than one 
problem in a given field of interest has been located as an 
outcome of the type of reading and thinking suggested in 
preceding sections of the present chapter and in Chapter III, 
or as a result of direct field contact, obviously the university 
investigator, field research worker, or teacher must decide 
which problem is to be attacked first. In most instances the 
controlling point of view in selecting a problem should be that 
of overcoming obstacles which prevent education or the school 
from performing its functions or attaining sound objectives, 
rather than grasping at any problem which turns up, in which 
there may be only artificial or transient interest. Unfortu- 
nately, many graduate theses fall in the latter category. Mere 
collection of data should not be an end in itself. However {in 
isolating and identifying any type of problem, pure or applied, 
a thorough knowledge of the related literature is essential. 

If responsibility for choosing the problem rests primarily 
on the student or field worker rather than on a member of the 
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university faculty or supervisor of field research, greater in- 
terest and intellectual growth should result. As a matter of 
fact, in most graduate fields, the ability of the student to 
select, plan, and complete a study is considered the best single 
measure of competence in the area represented. It is true that 
professors are usually willing to weigh with students the merits 
of suggested problems. This type of insight into educational 
problems is welcomed by progressive administrators, super- 
visors, and directors of school research, and such officers fre- 
quently play a réle in encouraging field research similar to 
the service rendered by university professors in stimulating 
graduate students to work out their theses. 

The subsequent discussion of the selection of a problem for 
investigation may be based on seven criteria suggested by 
Reeder: * (1 novelty of the problem, (2) the investigator’s 
interest in the problem, ( a practical value of research on 
the problem to the student or investigator and others, (4) 
the worker’s special qualifications to attack the problem, (5) 

“availability of data on the problem, (6)Cost of investigating 
the problem, and (7)“time probably required for the investi- 
gation of the problem. 

The novelty of the problem may be determined through a 
bibliographical canvass of available related studies, as out- 
lined in Chapter III. If this survey is neglected in the selec- 
tion and definition of the problem, the investigator may’ be 
attacking a problem on which evidence is already available. 
Certainly the investigational attitude and library technique of 
the student who submitted a manuscript to Thorndike with 
the hope that it would be accepted as a Doctor’s thesis are 
not to be recommended. This candidate for a graduate degree 
said, in effect, “I know this thesis is original because I have 
read nothing on the subject.” °° 

The survey of related studies not only aids in preventing 

59 Ward G. Reeder, How to Write a Thesis, pp. 23-6. Bloomington, Ill.: Public 
School Publishing Co., 1930 (revised). 


is 60 W, A. McCall, How to Experiment in Education, p. 8. New York: The Macmillan 
0., 1923. 
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duplication of investigations previously conducted by other 
workers, but may uncover additional problems for solution, 
assist in the choice of research procedures, and provide a sum- 
mary of related investigations, which is usually considered an 
essential part of a research report. A more detailed discussion 
of the problem of duplication may be found in Chapter III, 
but it seems desirable to give instances of mistaken concepts 
of the meaning of duplication here. Community and environ- 
mental differences vary to such an extent that the solution of 
a problem for one area will not hold for another. School build- 
ings in Florida or California differ in some respects from those 
in Maine, Minnesota, or Canada. The problem of the all-year 
school in a mountainous coal-mining region is not the same as 
in the select residential section of a large city. The secondary- 
school curriculum of agricultural Iowa varies in some respects 
from that of industrial Detroit. Sometimes earlier studies need 
to be repeated for verification or to survey current conditions 
at stated intervals. 

The question of novelty or newness has an angle other than 
that of duplication of earlier work. The study should employ 
as nearly as possible the most recent data. Too frequently 
students who pursue their graduate training during summer 
sessions collect data for a thesis, then allow the material tc 
lie fallow for two, three, or even more years. Fallacious con- 
clusions result from such data when they are applied to the 
present. This is especially true when the material has been 
collected for several years prior to the present writing, in 
the early stages of an economic depression which kad not then 
touched the schools directly. For example, data collected in 
1930 and presented in 1934 or 1935, concerning summer ses- 
sions in public schools, high-school annuals, salaries of athletic 
directors and teachers of physical education, and an occupa- 
tional analysis of young high-school graduates, are far from 
accurate in the light of the great social and economic changes 
of the intervening years. 

In terms of the criterion of interest, it should not be difficult 
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in education, an applied science, for student or teacher, for 
university or field investigator, to find in his own area of work 
a number of practical problems of real concern. In fact, the 
story of “The Great Stone Face” has a direct application in 
the case of feld workers or students who look everywhere 
except in their own “back yards” for pressing problems of im- 
mediate interest. A major reason for encouraging workers to 
select their own research problems is that the results should 
prove better in terms of the growth of the investigator and 
the quality of work done. 

Of course, in terms of the criterion of practical value, the 
solution of the difficulties suggested in the preceding para- 
graph is of greater present worth to the worker and to educa- 
tion than to attack “pure” research, which may have little or 
ao immediate application to school procedure. However, the 
foregoing statement should not be interpreted to mean that 
so-called “pure” research is to be neglected, since in many 
cases the pure research of to-day provides the basis for the 
applied research of to-morrow. For example, Franklin’s work 
with electricity was not applied extensively until more than a 
century later when Edison did his great work; nevertheless it 
was essential to progress in this field. Possibly the adoption of 
this point of view would promote more widespread tolerance 
toward investigations of animal learning, ball-tossing, target 
shooting, maze learning, memorization of nonsense syllables, 
etc: 

To obtain the special training needed by the investigator to 
attack a given problem, ordinarily he may become better ac- 
quainted with his area of interest through courses pursued, 
actual field experiences, or careful study of the available lit- 
erature, preferably through the three avenues combined. Many 
problems involve the use of special techniques—historical, ex- 
perimental, statistical, or testing procedures—which require 
special training on the part of the investigator. These tech- 
niques of investigation are discussed in subsequent chapters. 
Though it is preferable to secure the appropriate training 
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before selecting a problem for investigation, field workers 
and students often find it necessary to pursue correlated read- 
ing, study, or even courses after the exact nature of the prob- 
lem and its difficulties have been determined. Examples of 
the failure to secure needed training and the results of such 
lapses may be given. With no previous contact in the field of 
testing, a young candidate for the Master’s degree adminis- 
tered and scored mental tests in such a manner as to report 
children with 1.Q.’s below 20 in the fifth grade, Examination 
of test scores secured by experienced workers revealed the 
extent of the error, although even an elementary knowledge 
of mental tests is sufficient to show the fallacy of the original 
results. In studying the educational development of a higher 
institution, another worker with inadequate training in history 
and the historical method had great difficulty in making his 
report anything more than an encyclopedia of events chrono- 
logically arranged and neglected problems of interpretation, 
cause and effect, and the influence of a variety of external 
forces on the development of the institution in question. 
Probably one of the most important abstract personal quali- 
fications is the ability to suspend judgment and to maintain a 
detached, impersonal, unbiased point of view. Frequently, on 
the basis of a subjective or emotional reaction, the immature 
worker makes extreme statements concerning the relationships 
between nature and nurture, or education and good govern- 
ment, not recognizing the reciprocal play of forces in such 
situations, When a strongly religious teacher in a denomina- 
tionally controlled secondary school learned that the majority 
of such teachers were required by their boards to be church 
members, and that this was not the case in public high schools, 
he hastily concluded that the character of the former teachers 
was correspondingly better, failing to recognize that church 
affiliation and good character are not necessarily synonymous. 
Preparing an analysis of a county-seat school population in 
Ohio, an enthusiastic native son described the integrity of the 
people, the local industries, and the beauty of the landscape 
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in terms that would have put all the adjectives of the real 
estate promoters to shame. Members of certain races, in dis- 
cussing the educational problems of their people, frequently 
experience great diffculty in maintaining a detached, scientific 
point of view, and in many cases frankly admit this limitation 
when it is pointed out. An ardent advocate of the activity 
movement, only after strenuous self-discipline, was able to re- 
port the transition of his school system to the activity plan 
in the dispassionate terms of science rather than those custo- 
marily used before the local Boosters’ Club. A small-town 
superintendent, with a janitorial force of three in whom he 
took great pride, sought to generalize concerning janitorial 
loads on the basis of their performance, failing to keep in 
mind the fact that the heights of a giant of eight feet, a dwarf 
of four feet, and a midget of three feet do not mean that the 
average height of man is five feet. 

There are happily many other instances of investigators who 
maintain a detached point of view, in spite of strong condi- 
tioning factors. Of two doctoral candidates with ministerial 
training, one had studied the learning behavior of a gorilla 
and of children, but could not be persuaded into making com- 
parisons unwarranted by the data at hand. The other had 
investigated the effect of nicotine on white rats and had rather 
strong convictions concerning the use of tobacco, but was 
totally unwilling, without experimental evidence, to generalize 
his findings to apply them to human beings. 

Other violations of valid procedure, caused partly by gaps 
in the training of the worker, will be cited in the several chap- 
ters dealing with investigational techniques. 

Beginners in research, especially graduate students, fre- 
quently fail to consider carefully the criteria of availability of 
data, cost, and time required in working out a proposed study- 
A doctoral candidate undertook a study of teacher trainìng 
which he realized later could not be solved without visits te 
the teachers’ colleges of the country. He could not afford the 
expenditure of money and time required, so it was necessary 
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to take up another problem, a detailed cost study of a group 
of near-by high schools, where the factors of availability of 
data, cost, and time required were in keeping with his re- 
sources. Another worker proposed a detailed analysis of the 
Negro secondary-school population in a large city. He utilized 
a case-study procedure, without giving due consideration to 
the fact that a few months later he would be employed hun- 
dreds of miles away where the sources of the data would be 
relatively inaccessible. Over many months two doctoral candi- 
dates collected curriculum materials, presumably as the basis 
for courses or units of work in the social studies, dealing with 
the traits of a good citizen and the topics treated in the 
Literary Digest respectively, without foreseeing the probable 
need for testing out such materials in a learning situation and 
without arranging for either the facilities, time, or funds for 
such testing. 

Too frequently the Master’s candidate has a “questionnaire 
complex” which closes his eyes to other available sources of 
data. Desirable exceptions are found in the case of workers 
who go to the records in the appropriate files or use the inter- 
view, where feasible. For example, an investigator studying 
teacher tenure in a limited area of Indiana went to the offices 
of the school superintendents involved instead of circulating 
questionnaires among teachers. Another Master’s candidate 
interviewed newspaper editors in a group of states with respect 
to their policies concerning space devoted to school items, 
since a questionnaire probably would have accomplished little. 
Another student proposed a questionnaire canvass of the atti- 
tudes and interests of school-board members, when he had im- 
mediately available data on the way in which a foundation 
educational program would work out in his own and adjoining 
counties. 

To comment specifically on the time required for the comple- 
tion of a graduate thesis, it will be recognized that the variable 
factors are the student, the problem, and the institution, Few 
good Doctors’ dissertations are completed within less than a 
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year, and most such studies are in progress over a longer 
period of time. Sometimes the Master’s thesis has been worked 
out within a period of three or four months, when full time 
could be devoted to the project, although this is the exception 
rather than the rule. Such facts suggest the desirability of early 
selection of the thesis problem by the graduate student and of 
deliberate, careful procedure on the part of both university 
and field workers. 

Numerous studies of large scope, recently investigated, 
which stand the test of the foregoing criteria exceptionally 
well, are described in the Department of Research News of 
the Journal of Educational Research. For example, certain of 
the newer types of research reviewed there within the two 
years preceding this writing deal with the relation between the 
secondary school and the college, development of qualitative 
standards for secondary schools and colleges, the effect of 
motion pictures on youth and the development of appropriate 
standards of appreciation, the effect of the radio on children, 
relations between the emotions and the educative process, ex- 
perimentation in character education, child development, edu- 
cational diagnosis, and the social studies in the schools. In 
acquiring a better understanding of the criteria previously dis- 
cussed, it may be of some value to the reader to apply them 
to such investigations as are outlined briefly in numbers of 
the Journal of Educational Research and of certain other 
periodicals current at the time this paragraph is read. 

The foregoing discussion of criteria for selection of a 
problem may be supplemented by examination of a similar 
set of standards originally intended for workers in the social 
sciences: ®t 

1. Does the field appeal to my interest? 

(a) Is the interest purely intellectual? 

(b) Is the interest present because of reward—pecuniary returns, 
ey of advancement in position, increased authority, and so 
ort! 


01 W. C. Schluter, How fo Do Research Work, pp. 11-2. New York: Prentice-Hall 
1926, 
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2. Are the results that may be obtained of practical or utilitarian 
significance? 
(a) May they be of use in business? 
(b) May they be of use to society, to government, or to others? 
3. Does the field present gaps in verified knowledge which need to 
be filled? 
4. Does the field require reworking? 
5. Does the field permit extension of inquiry beyond the present 
limits of verified knowledge? 
6. Is the field pivotal or strategical from the standpoint of the 
immediate purposes which the possible results of investigation are to 
serve? 


Another writer emphasizes three major procedures as aids in 
choosing administrative problems: (1) acquire the attitude of 
questioning every administrative procedure in education, and 
inquire what evidence there is to support one procedure in 
preference to another; (2) really know the most important 
fifty studies in educational administration; and (3) chart the 
issues which superintendents of schools must meet, list the 
kinds and amounts of knowledge, and estimate the reliability 
of the knowledge available. In more specific terms he presents 
a list of standards to which the thesis problem in educational 
administration at Teachers College, Columbia University, 
should conform. Such standards (which follow) are equally 
applicable in other subdivisions of education, whether the given 
problem is in the university or field. 


1. Must be within the field of educational administration. 

2. Must be unsolved, so that its solution will be a “contribution to 
education.” 

3. Must be real in the senses that its solution is a felt need and 
that it is capable of fairly definite solution based on adequate actual 
data. 

4. Must make a worth-while or significant contribution to education. 

(a) Discover additional facts or new practices. 
(6). Substantiate questioned facts, theories, or practices. 
(c) Perfect a previously discovered technique. 
5. Must be definite in that its conclusions have real value. 
62 Carter Alexander and Others, Educational Research, pp. 2-6. New York: Bureau of 


Publications, Teachers College, Columbia University, 1927, Also see later revisions of 
this guide. 
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6. Must be sufficiently limited to permit of exhaustive treatment. 
7. Must be of sufficient value to justify the effort and time em- 
ployed in the research. 


An interesting chart ® has been evolved for making a com- 
parative study of two or more research problems and covers 
many of the items discussed above. Parallel columns contain 
the analysis of the two problems compared. A second part of 
the chart, not here reproduced and not fully worked out, deals 
with evaluation and considers unusualness, scientific signifi- 
cance, practical significance, and timeliness. Such an outline 
should be of considerable assistance in determining which of 
two or more problems is most feasible, although it cannot be 
applied in a mechanical manner or with a high degree of ob- 
jectivity. At least it serves the purpose of directing the atten- 
tion of the student and critic to items of importance in the 
selection, planning, and evaluation of research studies. 

Formulating and stating the problem. After the problem 
has been selected, it must be definitely formulated and stated. 
McCall describes three types of investigators in terms of the 
way in which they formulate their problem." The first type of 
worker suggests problems so broad in scope and with so many 
possible variables involved that it is virtually impossible to 
plan the details of the investigation. For example, a would-be 
experimenter proposed the problem, or rather, field of investi- 
gation: “What is the effect of various factors upon learning?” 
Apparently, no recognition was taken of the fact that psy- 
chology reveals many factors affecting learning, and that there 
are various types of learning. Later the statement of the prob- 
lem became: “What are the effects of distribution of time 
upon learning?” Obviously, there are hundreds of ways in 
which to distribute time. Finally the problem became: “Will 
a typical fourth-grade class in silent reading, spending three 
thirty-minute periods per week, accomplish more or less than 
an equivalent class spending five periods of eighteen minutes 


63C. W. Hall, “Comparing Research Problems,” Educational Research Bulletin, IX 
(September 10, 1930), 339-42. 
64 W A. McCall, op. cit.. pp. 9-11. 
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CHART FoR Comparative Stupy or Two or More RESEARCH PROBLEMS 


Title or Field 


Problem No, 1—Social 
Intelligence 


Problem No. 2—Extra- 
curricular Activities 


Part I. Availability 


Statement of Problem... 


Delimitation 


Principal Assumptions to} 
Be Made 


‘Survey of Literature: 
1, Extent . 


Delimita-| 


2. Possible 
tion 


The construction and 
trial of a test to 
measure social intel- 
ligence. 


. By social intelligence 
is meant the ability 
to get along with 
other people. 

. Test will be designed 
for Freshmen enter- 
ing college. 


. That there is a gen- 
eral ability which 
conditions the ability 
to get along with 
people. 

. That participation in 
extra-curricular ac- 
tivities is a criterion 
of this ability. 


It is necessary to be 
at least slightly 
familiar with the en- 
tire literature of per- 
sonality. | Roback’s 
Bibliography (1927) 
lists about 1,500 
titles. Many articles 
have appeared since. 


B. For intensive reading 
only the experimen- 


What is the correlation 
between extracurricu- 
lar activity in high 
school and extracur- 
ricular activity in 
college? 


. Would study only 
that group which 
reach the senior year 
in college. 

. Would consider only 
those activities which 
are recorded. 


. That extracurricular 
activities are essen- 
tially the same in 
high school and col- 
lege. 

. That conditions 
which determine the 
extent of this activity 
are similar in high 
school and college. 


. The general literature 
is fairly voluminous 
and quite recent. It 
is mostly on a sec- 
ondary-school level. 


B. If attention is limited 


to the experimental 
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Title or Field 


Job Analysis .........- 


Analysis of EA ai 


to Be Used . 
Ry Philosophical . 
2. Historical ... 
3. Descriptive 
4. Analytic . 
5 


« Deductive ....... 


Problem No. 1—Social 
Intelligence 


Problem No, 2—Extra- 
curricular Activities _ 


tal studies need be 
included. This re- 
duces the number of 
titles to about 300. 


A. Analysis of literature 
B. Collection of test 


items 
1. By sampling the 
literature 
2. By interviewing 
individuals 
(a) College fac- 
ulty 
(b) High-school 
teachers 


(c) Student body 


C. Construction of test 


with large number 
of items 


D. Setting up criterion 


and giving test 


E. Selection of items by 


partialing out intel- 
ligence and scholar- 
ship 


F. Final trial and 


standardization 


Analytic —in determin- 


ing the factors that 
should enter into the 
test 


Experimental—in for- 


mulating test items 
and in trying them 
out on criteria 


Statistical—in evaluat- 


ing items and com- 
plete test. Use partial 
correlation technique 


work on the college 
level, the number of 
titles does not exceed 
twenty-five. 


A. Analysis of the litera- 
ture and correspond- 
ence with others 
working in this field 

B. Prepäration of a 
questionnaire to be 
given to all college 
Seniors which will se- 
cure record of both 
high-school and col- 
lege participation 

C. Devising a weighting 
scheme for scoring 
activities 

D. Correlation of high~ 
school and college 
scores 


Statistical techniques 
would be used in 
weighting, in scoring, 
and in running cor- 
relations 
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Title or Field 


Problem No. 1—Social 
Intelligence 


Problem No, 2—Extra- 
Curricular Activities 


8. Questionnaire 


Availability of Data....| / 


of Objectivity) 


Special Financing 
Summary Estimate of 
Availability ..... ABE 


. Data for test items 
present no great dif- 
ficulty. 

. Securing of adequate 
criteria will be most 
difficult, 


. Can be objective 
within assumptions. 

. Considerable difñ- 
culty as to criterion. 
May be able to 
quantify. 


. There may be no 
such separate ability. 

. All test material 
may correlate highly 
with intelligence. 


.Three months to 
read literature 

. Three months to se- 
cure test items 

C. Six months to evalu- 

ate items 

Six months for trial 

and write-up 


D. 


$200 to $300 (More will 
be required if Hol- 
lerith cards are 
used.) 


Probably unnecessary 


Tke questionnaire would 
be used to gather 
data. 


Some difñculty might be 
met in obtaining to- 
tal populations unless 
administrative inter- 
est is obtained. 


Questionnaire data can 
be tested as to relia- 
bility and handled 
statistically. Much 
more rigidly objec- 
tive than No. 1. 


Should get some kinds 
of usable results. 


A. Three months to read 
literature and prepare 
questionnaire 

. Three months to give 
and collect question- 
naire 

. Three months to 
handle data 


$100 for questionnaire 

$100 for traveling ex- 
penses if other col- 
leges are included 
(More will be re- 
quired if Hollerith 
cards are used.) 


Probably unnecessary 
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each per week?” It hardly requires further comment to show 
that the factors which affect learning, the various types of 
learning, the available educational levels, and possible varia- 
tions in length of practice periods are numerous, making it 
necessary to formulate the problem in more specific terms. 

To use the illustration which is employed in a variety of 
ways in other chapters, students sometimes say they wish to 
investigate “individual differences” as a thesis problem. It re- 
quires no comment to demonstrate that this is a field or area 
of almost countless problems, many large enough for a Master’s 
or Doctor’s study. However, to survey the ways in which a 
selected group of junior high schools adapts its program of in- 
struction to the individual differences and varying needs of the 
pupils involved can be delimited within reasonable bounds. 
Likewise graduate students sometimes state that they wish to 
attack the problem of “school administration” or of “science 
teaching” in working out a thesis, when in reality a field of 
many problems has been named. Their difficulty is the same as 
that of the student described by McCall. 

The discussion in the preceding paragraph may be amplified 
somewhat by presenting a list of problems which are formu- 
fated and stated within reasonable limits; some of these have 
been worked out as graduate theses. 

In a second list are examples of actual instances in which 
rather broad titles of Doctors’ dissertations have been narrowed 
with the passage of time, and with a realization of the com- 
“plexity of the large original topic selected. The original title is 
given in parentheses. 

On the other hand, rather narrow or localized problems are 
sometimes broadened as the result of experience with, and in- 
creased insight into, the given area of investigation. The third 
list provides illustrations of actual changes of this type in 
Doctors’ dissertations. The original title is given in parentheses. 

Ina fourth list are topics which, as stated, hardly lend them- 
selves to scientific attack. In some cases, only a large area is 
named; in others, a subjective, emotional, or oratorical reac- 
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tion is implied; and in still other instances, only a mass of raw 
data or of personal experiences is suggested. 


I. ILLUSTRATIVE PROBLEMS CAPABLE or FORMULATION AND STATEMENT 
Wirra REASONABLE LIMITS 


1. Inductive versus deductive methods of teaching fifth-grade 
arithmetic . 

2. Effectiveness of the “dualistic” versus the “coördinate” form of 
supervision in cities of 100,000 and over 

3. Liability of school boards for injuries to children while on 
school property 

4. The grammar-translation versus the direct conversational 
method of teaching first-year French 

5. Relation between mental and physical maturation between the 
ages 12-15 

6. The development of the Latin Grammar School in Massachu- 
setts from 1635-1750 

7. Practices of the forty-eight states with respect to statewide 
adoptions of textbooks 

8. Determination of the mental ability, as measured by intelligence 
tests, necessary for successful work in a teacher-training institution 

9. Relation between school marks and character ratings of pupils, 
made by teachers 

to. A survey of population trends and selection of future building 
sites in a city of 400,000 

11. Historical development of the Arizona public-school system 

12. The reliability of reaction time tests 

13. Punctuation usages in the written work of certain rural schoo} 
children 

14. A study of errors in beginning Latin 

15. History of Catholic education in Kansas, 1836-1932 

16. Personality traits as measured by the Association reaction-time 
test i 

17. Secondary education in Georgia, 1732-1858 

18. Changes in Thorndike intelligence-test scores at the end of the 
college course 

19. Objective determination of the content of sixth-grade arithmetic 

20. An analysis of the vocabulary load of twelve general-science 
texts 
II. ILLUSTRATIONS or THE Manner 1x Wuicu Broan ProsLems or FIELDS 


Have Bren More Narrowry DELIMITED 


1, “Visual Defects as Factors Influencing Achievement in Reading” 
{The Effect of Poor Vision on School Adjustment) 
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2. “The Scholastic Achievement of Sixth-Grade Children in Relation 
to Behavior Problems” (Achievement of Elementary-School 
Children in Relation to Behavior Problems) 

3. “A Decade of Controversy in American Institutions of Higher 
Learning” (A Decade of Controversy) 

4. “The Functions of Prairie View State College in the Development 
of Public Education for Negroes in Texas” (A Study of Negro 
Land-Grant Colleges) 

“An Experimental Investigation to Determine Whether Homo- 
geneous Grouping or Heterogeneous Grouping Insures Greater 
Growth in Academic Achievement” (A Study of the Relative 
Effectiveness of a Specified Plan of Homogeneous Grouping 
Compared with Heterogeneous Grouping) 

6. “The Effect of Marginal Headings on Comprehension” (The Ef- 
fect of Captions, Marginal Headings, etc., on Comprehension 
in Reading) 

“The Measurement of Achievement in Music Education in Grades 
4 to 6 and 7 to ọ Inclusive” (Diagnostic Tests in the Field of 
Music Based Upon an Analysis of the Content of the Subject) 

8. “School Adjustment as Revealed by Teachers’ Marks and Objec- 
tive Tests” (Factors Other Than Intelligence Which Influence 
School Achievement) 

9. “An Evaluation of the New York City Junior High School Phys- 
ical Fitness Tests” (Physical Accomplishment of Negro Junior 
High School Boys in Harlem) 


a 


IIT, ILLUSTRATIONS or THE Manner IN Wicu RATHER NARROW OR 
Locarizep ProgLems Have Been EXPANDED 


“A History of Higher Educational Work of the Church of God” (A 
History of Findlay College) 

“Colonial Schoolmasters, 1635-1775” (The Colonial Schoolmasters of 
Maryland, Virginia, and North Carolina) 

“Standards Reflected in Fifty Sets of Primary Readers Published in 
the United States Since 1850” (Content and Methods in Primary 
Readers Used in Texas Since the Civil War) 


IV. Incusrrarive Topics INVOLVING too LARGE an Area, AN UNSCIENTIFIC 
Procepure, ok Merery an Unassrumacatep Mass or Data 
OR EXPERIENCE 


School boards 

Teaching of Latin 

Adolescent and child psychology 
The learning of school children 
The supervision of schools 


Sa it 
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6. Historical and comparative education in Ancient, Medieval, 
and Modern times © 

7. School textbooks, a plea for free books in a time of depression 

8. Election of the county superintendent, an argument that in a 
democracy the voice of the people should rule 

9. Experiences of teachers in rural schools, a running stenographic 
account of personal difficulties encountered by roo teachers 

zo. Teachers are born and not made, reasons and arguments for 
and against i 

ir. “Back bone,” or the development of character, a series ot 
inspirational assembly talks to junior high-school boys 

12. The need for more school buildings in Cincinnati, an ardent 
plea for a building program over the next fifteen years, as advocated 
by the local school survey 

13. The school of the future, the hopes, ambitions, and aspirations 
of the teachers of to-morrow 

14. Creative youth, as described and recorded verbatim by running 
stenographic account and dictaphones 

1s. Teaching, the great adventure, an inspirational, stimulating 
essay on the great service to mankind of the teaching profession 

16. Equalization of opportunity in education, let the wealth of the 
state educate the children and future citizens of the state 

17. Preparation of effective assembly talks to be given by a dean 
of girls to adolescent girls in a junior high school 

18. School buildings, apparatus, equipment, and supplies 

19. School textbooks, the curriculum, and course of study 

20. Code of ethics for the teaching profession, written as the result 
of, and based on, thirty years’ personal experience as a teacher 

21. How to discipline pupils, a handbook of the methods used by 
the three most successful teachers in a large school system 


A second investigator is described by McCall as the “pot- 
hole” type who selects such narrowly delimited problems as to 
appear an experimental drudge and as failing to see any rela- 
tion between his isolated studies and the general pattern or 
major problems of education. Certain of the ball-tossing, 
target-shooting, and maze-learning experiments may seem, in 
the judgment of some educational workers, to belong to this 
class. 

The third type is represented by the investigator who formu- 
lates the major issue or general problem, then subdivides it 
into smaller problems or specific working units. According to 
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McCall, the solutions of these smaller or more specific prob- 
lems, when assembled in an integrated pattern, are contribu- 
tions to the solution of the major issue, which alone may have 
real meaning and value. 

Monroe and Engelhart emphasize that the problem should 
be stated as a question or in such form that the question to be 
answered is clearly indicated. It has been stressed that a topic, 
such as the title of a thesis, merely names the particular field 
in which the problem lies and does not provide’ a‘satisfactory 
statement of the problem itself. According to Monroe and 
Engelhart, the various forms available for use in the general 
statement of the problem are as given below.** Occasionally 
students mistakenly assume that they must use each of these 
forms in the statement of a single problem, which of course 
is ridiculous. The type of statement to be employed depends 
on the preference of the worker and the nature of the problem. 


Iı. A question or questions 

(a) A single question 

(b) Several questions 

(c) A single question followed by several sub-questions 
2. Declarative statement 

(a) A single statement 

(b) A single statement containing several phases 

(c) A series of complete statements 

(d) A general statement followed by subordinate statements 
3. Statement followed by restatement in the form of a question 
4. A statement followed by a series of theses 


Defining and delimiting the problem. Obviously, it is not 
enough to formulate the problem briefly in the form of a ques- 
tion or declarative statement such as those listed above. More 
detailed definition and delimitation are necessary. Of course, 
the worker is at liberty to use as many procedures in defining 
and delimiting his study as make for clarification. The formu- 
lation and testing of hypotheses, as related to problem solving 
in education, are discussed in Chapter IV. Monroe and Engel- 


65 Walter S. Monroe and Max D. Engelhart, The Techniques of Educational Research, 
P. 14. University of Illinois Bulletin, Vol. XXV, No. 19. Bureau of Educational Research 
Bulletin, No. 38. Urbana, Ill.: University cf Illinois, 1928. 
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hart have provided an excellent discussion of the considerations 
which enter into the definition of the problem.” 


To define a problem means to specify it in detail and with pre- 
cision, Each question and subordinate question to be answered is to 
be specified. The limits of the investigation must be determined. Fre- 
quently, it is necessary to review previous studies in order to determine 
just what is to be done. Sometimes it is necessary to formulate the 
point of view or educational theory on which the investigation is to be 
based. If certain assumptions are made, they must be explicitly noted. 

The definition of the problem affords a basis for the subsequent 
phases of educational research, It is the guide for the collecting of 
data. The data are to be analyzed, organized, and summarized so as 
to be most useful for answering the questions specified in the defini- 
tion of the problem, and the conclusion is merely a statement of the 
answers resulting from the investigation. 

‘The basic importance of the problem and its definition indicates that 
they should appear early in a report of educational research, However, 
it is not unusual to find reports in which a large amount of introduc- 
tory material precedes the statement of the problem. Occasionally this 
may be desirable, but frequently a critical reader wonders if the 
formulation of this introductory material did not precede the formula- 
tion of the problem. 


Various ways of defining the problem are given by the same 
authors as follows: 


. Analysis of the major problem or problems in terms of subordi- 
nate problems 
2, Statement of the limits or scope of study 
3. Orientation of the problem 
(a) A historical account, remote or recent 
(b) A survey of previous studies or related studies 
(c) An analysis of previous studies or related subjects 
(d) Preliminary survey 
4. Description of the general nature of the problem 
(a) Type 
(b) Source 
(c) Procedure 
Statement of limitations of technique employed 
Recognition of assumptions and implications 
Importance, value, or significance of study to education 
Definition of terms 


Sx on 


66 Ibid., Chapter II. 
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Concluding statement. The content of this chapter, along 
with the information presented in Chapters III and IV, should 
provide the student or field worker with a reasonably adequate 
background in the selection, statement, and delimitation of a 
problem. Acquaintance with current educational literature and 
field contacts with teaching, supervisory, or administrative 
problems reveal many issues which require additional investi- 
gation. The types of available analyses considered most help- 
ful in locating problems or in determining whether the 
solution of a given difficulty is already in print include: (1) 
frequency distributions and topical analyses of the research 
completed in various fields, (2) authoritative statements of 
accomplishment and research completed, (3) analyses of edu- 
cational trends, (4) critiques, (5) educational prophecies, (6) 
surveys of research under way, and (7) statements of needed 
research. In addition, something may be said for McCall’s sug- 
gestion that one start an investigation and watch problems 
“bud” out of it.°* This is, however, a practice somewhat more 
appropriate for the laboratory than the classroom where the 
regular work of the school day or year must be done, without 
too much distraction for teacher and pupils. In other words, it 
seems more practical and economical in the classroom to start 
with a definitely planned study, intended to solve a specific 
problem, than to experiment at random because of an intel- 
lectual curiosity to see what related issues will become ap- 
parent in the course of the investigation. 

It has been stressed that in choosing a problem the investi- 
gator should consider the factors of novelty, interest, practical 
value, personal qualifications and expertness, availability of 
data, cost, and time. Without question, the problem must be 
delimited within reasonable bounds, if it is to be attacked 
scientifically. 

Certainly the investigator need not hesitate to select and 
delimit a practical field problem for study because it may - 
seem to him small and insignificant as compared with the 


67 W. A. McCall, op. cit, pp. 7-8. 
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multitude of unsolved educational problems illustrated and 
listed in this chapter. Rarely is a major discovery made which 
brings about sweeping changes in the complex processes of 
learning and teaching. Frequently, it is by a combination of 
results from large numbers of smaller studies that generaliza- 
tions are reached and investigational findings applied to edu- 
cational practice. 


We are familiar in mathematics with the doctrine of the summation 
of infinitesimals. The result is a finite magnitude whose size may indeed 
be very great. Each method, which through patient investigation is 
proved to be a little superior to the methods hitherto in operation, 
is one of the small quantities entering into this summation. Let us 
not suppose that any one of these procedures must produce a large 
effect in the complex learning process. Let us be prepared to accept 
small gains and to combine them for the purpose of producing large 
gains. Let us work bit by bit toward the master method. Let us play 
the educational game as the chess masters are now playing their game. 
Let us be content with an accumulation of small advantages. 


PROBLEMS AND EXERCISES 


r. Select a subdivision of education of major interest to you, and list 
at least six problems requiring investigation which have been em- 
phasized in the literature within twelve months preceding the 
undertaking of this assignment. 

2. Choose one of the problems named in working out the preceding 
assignment, and: 

(a) State, delimit, and define it in accordance with principles de- 
veloped in this chapter. 
(b) Analyze it in terms of criteria for choosing a problem. 
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units, fuel and heating problems, and maintenance and fixed charges. 
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ton Mifflin Co., 1930. Discusses the nature and sources of problems, 
and criteria for selecting problems. 

Asusaucu, E. J. “Research in School Administration,” Journal of Edu- 
cational Research, XXVIII (September, 1034), 48-9. A thoroughly 
codrdinated series of studies is advocated which will attempt to ar- 
ticulate the present situation in school administration and the future 
development of a fundamental philosophy, so as to make for con- 
sistency both in the training of administrators and in the performance 
of their duties. 

Arwoon, W: W. “Research and Educational Work in Geography,” Journal 
of Geography, XXVII (October, 1928), 263-70. Points out where 
geographical research is necessary, and describes the type of person 
needed. Believes geography has the possibility of contributing more 
to the establishing of good will among the people of the world than 
any other subject. Urges the founding of one or more institutions for 
geographical research in the different countries. 

Aurner, R. R. “The Status of Research in the Field of Business English,” 
Balance Sheet, XV (November, 1033), 111-6, 144. 

Barr, A. S. “On the Timeliness of Research,” Journal of Educational 
Research, XXVIII (September, 1034), 49-52. The difficulty with re- 
spect to the timeliness of research arises out of a difference of opinion 
between university workers and field workers as to the problems 
worth studying. 

| , and Dovctass, Lois. “The Pre-training Selection of Teachers,” 
Journal of Educational Research, XXVIII (October, 1934), 92-117. 
Summarizes recent theories, practices, and investigations; suggestions 
for further study and extensive bibliography. 

Barton, J. W. “Trends in the Junior College Curriculum,” Junior College 
Journal, V (May, 1935), 405-18. Despite ample warning, the junior 

| college remains predominantly a school preparing for continuation 
work, although there has been a decided emphasis on vocational and 
pre-professional education, which may be preparatory. Semi-profes- 
sional courses are on the increase in most junior colleges, and in 
scattered areas adult education and orientation, survey, or guidance 
courses are being introduced. 

Berwarp, L. L., and Bernarp, J. S. “A Century of Progress in the Social 
Sciences,” Social Forces, XI (May, 1933), 488-505. The development 
of the science of history and historical method, historical research, 
political science, political economy, professional economics and com- 
merce, sociology, social-science organizations and publications, gradu- 
ate social sciences, and the expansion of research. 

* Brxrer, H. H. Check-Lists for Educational Research, Chapter II. New 
York: Teachers College, Columbia University, 1928. Selection of the 
problem, preparation of the bibliography and survey of the literature, 
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formulation of the problem and outline of the procedure, and exami. 
nation of the proposed sources of data. 

Bope, B. H. “The Most Outstanding Next Steps for Curriculum Makers 
in the United States,” Teachers College Record, XXX (December, 
1928), 179-91. The most important next step in curriculum construc- 
tion is the application of such a conception to the selection and or- 
ganization of subject-matter, as to make the curriculum contribute 
to the disposition and power to “see life steadily and see it whole,” 
in terms of the circumstances and opportunities of the modern age. 

“The Next Step in Elementary Education,” Education, LV (May, 
1935), 513-7. Emphasizes the next step as a more adequate recog- 
nition of the doctrine that education means the continuous reconstruc- 
tion of experience, and of the need for interpretation in the intel- 
lectual and spiritual life of individual pupils. Acceptance of this respon- 
sibility will have a direct bearing on selection of subject-matter, on 
the progressive differentiation of the curriculum into subjects, and on 
the transformation of our entire educational system into an instru- 
mentality for social progress. 

Boynton, P. L. “Guidance—A Science or a Philosophy?” Junior-Senior 
High School Clearing House, VITI (May, 1934), 517-20. One who 
uses either science or philosophy to the exclusion of the other is 
seeing only one arc in the guidance program instead of the whole 
circle of related aspects. 

Brappury, Dorothy E., and Sxrers, Esther L. “An Analysis of the 
Literature Dealing with Nursery Education,” Child Development, 
VI (September, 1935), 227-30. 

Brwweman, R. P. “Ten Years’ Progress in Parent Education,” Childhood 
Education, VII (January, 1931), 249-56. Describes the work of na~ 
tional organizations, official state programs, child development research 
centers, college and university extension courses, local voluntary or- 
ganizations, and of the National Council of Parent Education. 

Briscotr, W. S. “Next Steps in Curriculum Organization—What and 
Why?” California Journal of Secondary Education, XI (February, 
1936), 107-9. 

Brooxs, F. D. “Mental Development in Adolescence,” Chapter V in 
“Mental and Physical Development,” Review of Educational Research, 
VI (February, 1936), 1-152. 

Browne, A. D. “Suggested Subjects for Physical Education ‘Theses and 
Dissertations,” Research Quarterly of the American Physical Educa- 
tion Association, II (May, 1931), 128-36. 

Brownett, W. A. “The Growth and Nature of Research Interest in 
Arithmetic and Reading,” Journal of Educational Research, XXVI 
(February, 1933), 429-41. Educational science must concern itself with 
“depth” rather than “extent” and “spread”; tabulation of studies in 
reading and arithmetic shows many to be by one-study authors who 
fail to publish the results of, or to carry on, further investigations, 
in the areas represented, 

Burton, W. H. “Probable Next Steps in the Progress of Supervision,” 
Educational Method, IX (April, 1930), 402-6. Scientific attack upon 
such problems as: a 
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x. The nature of the administrative organization of the school system, 
with especial reference to the provision for supervision 

2, The determination of more objective evidences of teacher efficiency 

3. The progressive development of a new point of view in teacher 
training and particularly the recognition on the part of supervisors 
and principals of their share in teacher training. 

Catpwert, O. W. “Research and Elementary Science Teaching,” Science 
Education, XVIII (April, 1934), 65-7. 

CARTWRIGHT, M. A. Ten Years of Adult Education: A Report on a Decade 
of Progress in the American Movement. New York: The Macmillan 
Co., 1935. Pp. 220. 

Cassipy, Rosalind, and Others. “Physical Education Trends and Progress,” 
Journal of Health and, Physical Education, VI (February, 1935), 
14-21, 62. 

Cuameprrs, M. M. “A Glimpse into the Future of School Administration,” 
Nation’s Schools, XIII (January, 1934), 13-4. Suggests as fields of 
research personnel administration in the schools and external educa- 
tional administration. 

Cuampers, M. M. “Two Major Issues in State Administration,” Nation’s 
Schools, XVII (April, 1036), 25-6. 

* Corr, Luella. Psychology of the Elementary School Subjects, Chapter X. 
New York: Farrar and Rinehart, 1934. Discusses possible studies of 
objectives, textbooks, teaching method, difficulty, children’s interests 
and abilities, and test construction. 

Corr, R, D. “A Plea for More Experimental Work by College Depart- 
ments of Modern Foreign Languages,” School and Society, XXXV 
(April 9, 1932), sor-3. Urges that the 123 problems listed in the 
Coleman report of the Modern Foreign Language Study be given con- 
sideration in the selection of theses and dissertations for graduate 
study, as well as made subjects for investigation within college de- 
partments. 

Cooper, Hermann. “A Forward Look in Teacher Education,” Education, 
LVI (December, 1935), 195-200. 

Coxe, W. W. “Frontier Issues of Educational Research,” Journal of 
Educational Research, XXVII (April, 1934), 561-3. Educational 
theory has advanced unevenly in the various fields of endeavor, with 
the result that practice is a peculiar blend of scientific findings and 
conventionalism. In many cases studies show a critical investigation 
of one phase and a naive acceptance of very closely related problems. 
Workers should extend the educational research frontier in two direc- 
tions, first, to investigate the unquestioned assumptions which under- ` 
lie much of the current practice, and second, to work on those prob- 
lems which lie in the borderland between education and allied fields. 

* Crawrorp, C. C. The Technique of Research in Education, Chapter I. 
Los Angeles: University of Southern California, 1928. Discovery of 
problems that need solving, and judging the relative values of those 
discovered. 

Curtis, F. D. “Some Reactions Regarding the Published Investigations in 
the Teaching of Science,” School Science and Mathematics, XXVIL 
(June and October, 1927), 634-41, 710-20. Impressions gained from 
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an analysis of learning and curricular studies on the teaching of sci- 
ence; mentions some of their more striking values and defects, and 
discusses briefly the present status of investigation in relation to 
future research in this field. 
Davıpson, P. E. “Curriculum Study, Present and Future,” California 
Journal of Secondary Education, XI (January, 1036), 27-32. 
Davis, R. A. “Research and the Schools,” Journal of Educational Research, 
XXVI (April, 1933), 561-8. Notes some of the most obvious defects 
of research and the influence on education; indicates improvement 
which may be effected, the contribution of qualitative methods in 
philosophical, integrative, and historical research, and needed training 
in problem solving. 
Dickson, V. E. “Scientific Research in the New Education,” Educational 
Method, XIII (April, 1934), 337-48. 
Dovorass, A. A. “On the Problems of Secondary Education in California,” 
California Journal of Secondary Education, X (October, 1935), 415-20. 
1. The aims or “platform” of secondary education 
2. The development of a series of principles for constructing and 
judging units of instruction 

3. The production of new teaching materials 

4. The articulation of the revised curriculum with the higher insti- 
tutions of the state 

Doverass, H. R, “Does the Future of Secondary Education Depend Upor 
Statistical Research in Foundational Thinking ?” Education, LIII (Jan- 
wary, 1933), 284-9. Urges that there need be no conflict between 
Philosophy and science; since both are needed to organize thinking and 
to point the way out. 

“Three Hundred Years of Method, in the Secondary School,” Edu- 
cational Administration and Supervision, XXI (May, 1035), 321-33. 
Discusses development of the field of individual psychology, use of 
visual and auditory aids, character education, the testing movement, 
controlled experimentation, pupil-initiative and responsibility, a coöp- 
erative and sympathetic pupil-teacher relationship, and psychology and 
method as essential in the training of secondary-school teachers. 

Durrett, D. D. “Research Problems in Reading in the Elementary School,” 
Elementary English Review, XIII (March, 1936), 101-6, 111. 

Ecxerserry, R.-H. “The Need for a Study of Comparative Education,” 
School and Society, XXXII (October 4, 1930), 455-6. To study for- 
eign education and practice would enlarge our horizon, throw new 
light on our problems, and break down prejudices. 

EpMonson, J. B. “Some Needed Changes in Secondary Education,” Uni- 
versity of Michigan School of Education Bulletin, VIL (January, 
1936), 52-4. h 

Eetts, W. C. “Status of the Junior College in the United States, 1935-36,” 
School and Society, XLII (February, 1936), 160-2. 

* EVERETT, Samuel, and Others. A Challenge to Secondary Education: Plans 
for the Reconstruction of the American High School. New York: D. 
Appleton-Century Co., 1935. Pp. viii + 354. 

*Frucet, J. C. A Hundred Years of Psychology. New York: The Mac- 
millan Co.. 1934. Pp. 384. A historical discussion of psychology since 
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l 1833, with the last chapter considering the present position of psy- 
chology, which is characterized as infantile and uncoördinated; sug- 
gests that the next thirty or forty years may be occupied with 
efforts toward consolidating the positions already won and further 
systematic advance on a unified front. 

Freeman, F. N. “Needed Research in the New Education,” Progressive 
Education, X (April, 1933), 220-4. 

“Proposed Educational Reforms—Some of the Major Issues,” Na- 
tion’s Schools, XII (July, 1933), 30-3. Since debates concerning pro- 
posed reforms in education are often unproductive, an attempt is 
made to analyze issues and to weigh considerations impartially, dis- 
cussing freedom, self-expression, interest, overt activity, drill, morale, 
discipline, social responsibility, and purpose, with emphasis on de- 
velopment of social purposes in the individual translated into socialized 
behavior. 

“The Contribution of Science to the Principles and Methods of 
Teaching,” Childhood Education, VI (April, 1930), 348-54. Describes 
briefly the various types of scientific investigation which have yielded 

` fruitful results, and presents a few recent typical studies bearing 

: upon the various methods and principles of instruction. 

Frencu, W. M, “A Century of Teacher Training in New York,” Educa- 
tion, LVI (December, 1935), 215-23. 

l Gamopritt, Bessie L. “A Critical Review of Researches in Supervision,” 
Educational Administration and Supervision, XV (April, 1929), 279-89. 
Suggests problems for investigation. 

* Goon, Carter V. How to Do Research in Education, Chapter V. Baltimore: 
Warwick and York, 1929. 

* Goopgnovucn, Florence L., and Anperson, J. E. Experimental Child Study. 
New York: The Century Co., 1931. Pp. 546. Contains many outlines 
of experiments to be made on groups of children. 

GoopyKoontz, Bess. “New Problems in Elementary Education,” Nation’s 
Schools, XVI (September, 1035), 23-26. Deals with: (1) need for 
reorganization, (2) effectiveness of coöperation, (3) need for restate- 
ment of functions, and (4) curriculum expansion. 

“The Scientific Method and Creative Supervision,” Educational 

Method, XIII (May-June, 1934), 385-90. 

Grirritn, C. R. “A Laboratory for Research in Athletics,” Research 
Quarterly of the American Physical Education Association, I (October, 
1930), 34-40. States that the field of athletic competition with its 
many problems has virtually remained untouched; describes the labora- 
tory for research in athletics at the University of Illinois in terms of 
its history and the problems studied. 

Harar, Henry. “A Critique of Public-School Courses of Study, 1928-20,” 

Journal of Educational Research, XXI (February, 1930), 109-19. 
In 242 courses of study the following elements were studied: typog- 
raphy; inclusion of procedure, objectives, and tests; form of the 
unit of work; provision for adapting instruction to individual dif- 
ferences; and other tendencies. 

—— “Next Steps in Curriculum Making,” Elementary School Journal, 
XXX (September, 1030), 16-24. 
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——“Next Steps in Curriculum Making for Home Economics,” Jour- 
nal of Home Economics, XXI (August, 1929), 570-6. The steps are: 
preparing an introductory statement of guiding principles, determining 
objectives, composing the units of school work, introducing reviews, 
and formulating tests. 

Henpricxsox, Gordon. “Some Assumptions Involved in Personality Meas- 
urement,” Journal of Experimental Education, II (March, 1934), 
243-9. Three basic problems are discussed: (1) What is the nature of 
personality? (2) How stable is personality? (3) What social factors 
are involved in personality measurement? 

Henmon, V. A. C, “Recent Developments in the Study of Modern Foreign 
Language Problems,” Modern Language Journal, XIX (December, 
1934), 187-201. 

———"The Function, Value, and Future of Educational Research in 
Colleges and Universities,” Journal of Educational Research, XXVII 
(March, 1934), 493-502. 

Hivexs, C. M. “The Next Quarter Century,” Mental Hygiene, XIX 
(January, 1935), 69-77. Deals with the work and future of the Na- 
tional Commission for Mental Hygiene, which is to direct its efforts 
of the next twenty-five years to promoting the happiness and efi- 
ciency of man through a new understanding of him as an organism, 
in which the physical and mental are parts of a whole, and to a new 
view of the individual as part of a social organization, 

Hissoxo, Mary C., and Hissonc, Clyde. “The Scientific Movement and 
Progressive Education,” Education, LI (September, 1930), 1-9, A com- 
parison of the new school and the traditional school, together with 
Suggestions for development of the child who can think for himself 
and investigate a problem thoroughly, stressing critical attitude and 
“systematic, orderly, and exact thinking.” 

Horxiys, L. T. “Prospects for Progress in Home Economics Education,” 
Journal of Home Economics, XXIII (February, 1931), 117-23. Deals 
with three main questions; What changes are coming about in the 
family as a social institution? What are the fundamental bases for 
family life? What are the opportunities of home-economics education 
in meeting present needs? 

Horz, H. G, “Trends in the Development of Secondary Schools,” North 
Central Association Quarterly, X (January, 1936), 287-97. 

Huwnarn, F. W. “Today's Youth Problems,” Journal of the National 
Education Association, XXV (January, 1936), 13-28. 

Huones, J. M. “Suggested Study on the Relation of Training of Teachers 
in Content Subject-Matter to the Effectiveness of Instruction,” Jour- 
nal of Educational Research, XIX (March, 1929), 228-32. A sug- 
gested procedure to determine to what extent college training in 
aR functions in the work of high-school teachers of mathe- 
matics. 

Houuurisn, H. G. “A Critical Problem for Philosophy of Education,” 
Educational Administration and Supervision, XVI (April, 1930), 241-8. 
Are we to continue to exist in the midst of change, taking our buf- 
fetings as they come, and be no little disturbed because the things 
once held dear seem denied us, or are we to orient ourselves to the 
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characteristic features of the times for the purpose of evolving a pro- 

gram which will see in change an opportunity to live a more in- 

telligent life? 

Hurcmnson, M. E. “Some Needed Research in the Teaching of Latin,” 
Classical Journal, XXIX (February, 1934), 335-56. 

Jastonower, Joseph, “Mathematics Teaching in the Next Ten Years,” 
Progressive Education, XII (May, 1935), 338-41. The three most 
important realizable outcomes of mathematics teaching are listed as: 
1. The notion of mathematics as a means of describing time and 

space aspects of the world of phenomena 

2, The notion that the concepts of mathematics have a history and 
are understood the more clearly in terms of their historical de- 
velopment 

3. The notion that mathematical work is often the occasion for 
making explicit the methods of reasoning 
In shaping the curriculum, guides may be found in the aims men- 

tioned above, which suggest that: 

1. Mathematics will get much of its material from the fields in which 
it is applied. 

2. The mathematical curriculum of the secondary school will in time 
show little of the rigid division among the various aspects of the 
field. 

3. Mathematics teaching in the secondary school will make the study 
of methods of reasoning a direct object of study. 

Jewerr, Ida A. “A Decade of Research in English in Teachers Colleges,” 
Teachers College Record, XXXV (March, 1934), 460-72. The studies 
are classified under the heads of teacher supply and demand, curricu- 
lum, classroom techniques, and measurement. 

Jupp, C. H. “A Century of Progress in Education,” Phi Delta Kappan, 
XVI (August, 1933), 40-54. Development since 1833 of compulsory 
professional training for entrance into teaching, history of the instruc- 
tional program, teaching of health, and methods of teaching. 

Problems of Education in the United States. Recent Social Trends 

Monographs. New York; McGraw-Hill Book Co., 1935. Pp. 214. Re- 

veals important current trends and presents some of the problems 

which inevitably arise in a social system that is undergoing rapid 

evolution. x 

“The Scientific Development of the Curriculum,” Elementary School 

Journal, XXXIII (April, 1933), 577-91. Defines the school as so- 

ciety’s agency for socializing individuals. Shows need for a new and 

more comprehensive view of the science of education, which must ally 
itself with the whole family of social sciences. This broad view sup- 

E plies the formula for curriculum reconstruction. 

Kerauver, G. N. “Proposals for a Program of Evaluation of Guidance,” 

School Review, XLII (September, 1934), 519-26. 

= Kaprer, Paul, “The Experimental Study of Education with Special Refer- 
4 ence to the Elementary School,” Journal of Educational Research, 

XII (September, 1925), 123-35. Points out a large number of prob- 

lems needing experimental study rather than decision by personal 

opinion; relates factors which have retarded the development of ex- 
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perimental education, and offers many reasons why we have not 
evolved a set of commonly accepted measuring scales. 

Kyrk, Hazel. “The Selection of Problems for Home Economics Research,” 
Journal of Home Economics, XXV (October, 1933), 680-6. A dis- 
cussion of criticisms that have been raised against advanced work 
in home economics, together with criteria for choosing problems for 
research. 

*Kytr, G. C. Problems in School Supervision. Boston: Houghton Mifflin 
Co., 1931. Pp. 214. 

* Loomis, A. K., and Others. The Program of Studies, pp. 278-80. Office 
of Education Bulletin No. 17, 1932. National Survey of Secondary 
Education Monograph No. 19, Washington: Office of Education. 
Certain major features of a new program of studies at the secondary- 
school level can be inferred from the trends which have been dis- 
covered, involving general plans of organization, guidance, emphasis 
on major fields, integration of work, the constant element of the 
program, and requirements for college entrance. 

Lyon, H. W. “An Elementary-School Principal Talks About Research,” 
Nation’s Schools, XII (August, 1033), 52-4. Says that the modern 
tendency to make the curriculum fit the individual has brought about 
a need for intensive studies of ways of recognizing and providing for 
differences, and discusses trends due to recognizing individual 
differences. 

*McCatt, W. A. How to Experiment in Education, Chapter X. New 
York: The Macmillan Co., 1923. A number of problems, submitted 
by graduate students, with a rough plan for solution indicated. 

McCuusky, H. Y. “A Neglected Laboratory for the Study of Youth,” 
University of Michigan School of Education Bulletin, I (December, 
1929), 33-4. The summer camp offers observation of all phases of 
living, since it permits child observation twenty-four hours a day; 
presents various types of social situations involving children of the 
same age; stimulates naive and spontaneous behavior; and permits 
simultaneous observations which are extremely difficult under ordi- 
nary circumstances. One’s view in the commonly used agency, the 
juvenile court, is fragmentary, pathological, and extreme. 

McConn, Max. “Examinations, Old and New: Their Uses and Abuses,” 
Educational Record, XVI (October, 1936), 375-411. 

Marss, Willa M. “Future Developments in Junior High School Mathe- 
matics,” California Journal of Secondary Education, X (December, 
1935), 547-50. 

Merey, E. O. “Some Problems in Measurement,” Education, LVI 
(November, 1935), 170-4. Deals with techniques for the study of 
individuals, techniques for the analysis and appraisal of learning 
activities, and measures of conduct and living. 

Merepira, H. V., and Sropparp, G. D. “Physical Growth from Birth to 
Maturity,” Chapter IV in “Mental and Physical Development,” Re- 
view of Educational Research, VI (February, 1936), 1-152. 

Meyers, Edna R. “Experimental Possibilities in the Public Schools,” 
Progressive Education, IX (January, 1932), 22-8. Describes the plan 
of an experimental school in Chicago, together with phases of the 
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program which can be incorporated in other schools with an enrolment 
of forty-five to fifty children. 

Morcan, J. E. “Twenty-five years of Prophetic Achievement,” Journal 
of the National Education Association, XX (May, 1031), 147-8. 
Points out the achievement of the National Recreational Association, 
its leaders, and its connection with the problem of leisure, wise use of 
which will do as much to determine national character in the next 
century as the common school has done during the past century. 

Mvers, A. F, “An Evaluation of Research on Teacher Preparation,” 
Journal of Educational Research, XXVI (November, 1932), 161-74. 

Norrsincer, F, R. “A Century of Progress in School Water Supply,” 
American School Board Journal, XCII (April, 1936), 30. 

Norrsrcrr, H. G. “One-third of a Century of Progress,” Junior College 
Journal, V (May, 1935), 395-494- Deals with growth in numbers and 
respectability, progress by experimentation and university leadership, 
functions of junior colleges, criticisms, and present development. 

Norton, J. K. “The Joint Commission Looks to the Future,” Journal 
of the National Education Association, XXIV (September, 1935), 
179-82. Counsels teachers that the future development of public 
education in the United States calls for a more realistic understanding 
by the teaching profession of the forces and groups which operate to 
determine educational policy; professional organizations must be better 
financed and more effective both as to program and methods than 
they have been in the past. 

Osrren, F. P. “A Problem for Coöperative Research—The High-School 
Dropouts,” School and Society, XXIX (May 18, 1929), 642-3. Sug- 
gests this problem, in view of the attention given pupils while they 
remain in high school, and that records should be available in high- 
school offices for preliminary information. 

Opett, C. W. “Education Measurement in the Secondary School,” Journal 
of Educational Research, XXVI (October, 1932), 81-9. 

*Qpum, H. W., and Jocuer, Katherine. An Introduction to Social Re- 
search, Chapter XXI. New York: Henry Holt and Co., 1929. Deals 
with “Exploring the Sources”: selection of the problem, defining and 
delimiting the subject, preparing for the study, obtaining data from 
primary sources, use of the library, bibliographies, note-taking, 
interviews, and correspondence, telephone, and telegraph. 

Otson, W. C. “Needed Research in the Prevention of Delinquency,” 
University of Michigan School of Education Bulletin, VI (April, 
1035), 102-3. An experiment, which should be inaugurated in various 
centers, is planned to demonstrate the effectiveness of the application 
of present knowledge in the prevention of delinquency; both con- 
structive and remedial programs should begin at an early age and 
continue over a protracted period. 

Ospurn, W. J. “An Outlook for Educational Research,” Journal of 
Educational Research, XXVIII (April, 1935), 623-5. The period of 
atomistic research is moving toward the organismic point of view. 

“What Next in School Administration?” Educational Administra- 

tion and Supervision, XXI (February, 1935), 107-14. The needed 

changes in school administration arising out of present conditions are: 
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x. Recognition of the multi-ordinal character of our fundamental 
functional concepts and the need for analyzing them 

2. Abandonment of additive and identification practices 

3. A new technical vocabulary of single-valued words or symbols 
as a basis for a new and more genuine science of school adminis- 
tration 

4. Definite knowledge objectively arrived at, showing at what levels 
our schools and colleges of professional education are actually 
operating at present 

Ons, A. S. “Fallacious Arguments Regarding Ability Grouping,” Child- 
hood Education, VIIL (December, 1931), 171-80. Maintains that the 
arguments used against homogeneous grouping are either fallacious, 
irrelevant, or an argument in favor of such grouping; discusses the 
meaning and fundamental principles of homogeneous grouping. 

Payne, E. G. “Research Problems and Trends in Educational Sociology,” 
Journal of Educational Research, XXV (April-May, 1932), 239-52. 

Procror, W. M. “Trends in Pupil Guidance,” California Journal of 
Secondary Education, X (January, 1935), 113-7. (1) Final vocational 
decisions not made in the junior high school, (2) increased emphasis 
on non-vocational phases of guidance in the junior high school, 
(3) relegation of vocational courses to the eleventh and twelfth grades 
and to the junior college, (4) correlation of school program and 
industry, (5) guidance for increased leisure, (6) emphasis on social 
engineering, 

Rankin, P. T. “Scientific Method in Supervisory Programs,” Educational 
Method, XIII (May-June, 1934), 391-3. Indicates the bases for judg- 
ing the degree to which a supervisory program is planned and 
conducted in accordance with the principles of scientific method. 

Reavis, W. C. “Secondary Education as a Field for Research,” Phi Delta 
Kappan, XII (June, 1929), 21-4. 

*Reeper, W. G. How to Write a Thesis, Chapter II. Bloomington, Ill.: 
Public School Publishing Co., 1930. Discussion of the selection, de- 
limitation and planning of the problem. 

Rrevet, F. A. “What, If Anything, Has Really Been Proved as to the 
Relative Effectiveness of Demonstration and Laboratory Methods 
of Science?” School Science and Mathematics, XXVII (May and 
June, 1927), 517-9, 620-31. Presents the status of the controversy, 
and criticizes and analyzes in some detail the studies in this field. 

Ropertson, C. G. “Educational Development, 1831-1931: A Centenary 
Survey and a Forecast,” School and Society, XXIV (October 3, 1931), 
445-58. Reviews the development of English education over a 
century and forecasts future trends; looks to psychology for aid, 
especially intelligence tests, in determining degree of educability; sees 
human activities viewed in terms of degree of trained intelligence 
and special aptitude required; expects functional differentiation be- 
tween sexes; stress on the end and purpose of education with a due 
place for religion. 

Rucc, Harold. “After Three Decades of Scientific Method in Education,” 
Teachers College Record, XXXVI (November, 1934), 111-22. States 
that in the initial stage of the movement the quantitative workers 
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have concentrated on increasing the objectivity of facts, have dis- 

played a meager interest in the validity of facts, and have devoted 

no effort at all to the questions of value, theory, and philosophy. 

Scientific inquiry has been assumed to consist essentially of the 

process of reducing error by buttressing quantitative measures. 

Ruccres, A. H., and Others. “Twenty-fifth Anniversary Celebration of 
the National Committee for Mental Hygiene,” Mental Hygiene, XIX 
(January, 1935), 1-37. The past achievements, present needs, and 
future hopes of the mental-hygiene movement. 

*Saprer, Michael. The Outlook in Secondary Education. New York: 
Teachers College, Columbia University, 1930. Pp. 56. 

“The Outlook in Secondary Education,” Teachers College Record, 
XXXII (June, October, December, 1930), 1-16, 123-39, 256-78. Says 
that when we find answers to questions concerning the progress and 
pitfalls of secondary education, youth and tests, and a liberal educa- 
tion, we shall be able to judge what tests of individual attainment 
are practicable without injury to the process of education itself. 

1. Whether education should be all-embracing or selective. 

2. To what extent the essentials of education must always come, 
not through the organized discipline of the school alone, but 
through agencies more intangible and atmospheric which kindle 
the zeal, strengthen the will, and color the purpose of those for 
whom the educational system is designed. 

3. What, in the province of national education, are the limits, if 
any, to the function of government ? - 

* ScuiuTer, W. C. How to Do Research Work, Chapters II, V, VI. New 
York: Prentice-Hall, 1926. Deals with selecting the topic, subject, or 
field of research; formulating or defining the problem; and differ- 
entiating and outlining the elements in the problem. 

Secer, David. “Recent Developments in Educational Measurement,” 
Education, LVI (November, 1935), 161-4. Deals with tests which seek 
to measure individual development, and discernment of proper rela- 
tionships between the various elements of life. 

Srvrrmp, J. E. Principles and Mechanics of Research, pp. 36 ff. Albu- 
querque, N. M.: University of New Mexico, 1935. Criteria for selecting 
a worth-while problem. 

Suurrtewortu, F. K. “A Decade of Progress in Methods of Measuring 
Character,” Journal of Educational Sociology, IV (December, 1930), 
233-41. Methods of measuring character are traced for a period of 
twenty-five years; lists five accomplishments during recent years 
as: (1) the rescue of measuring devices as useful instruments of 
evaluation with attendant improvements in the method of collecting 
data and a clearer recognition of the function of such tests; (2) 
measurement of moral knowledge; (3) a large number of ingenious 
suggestions, most of which need further study, for the application of 
pencil and paper testing to the measurement of special aspects of 
character; (4) measurement in ternis of conduct, especially of honesty, 
service, self-control, and inhibition; and (5) development and re- 
finement of the method of controlled observation, which promises 
to be important in the future. 
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Smits, D. E. Challenging Problems in American Schools of Education 
New York: Teachers College, Columbia University, 1935. Pp. 47. 
Smitu, Homer J. One Thousand Problems in Industrial Education. Minne- 
apolis, Minn.: University of Minnesota Press, 1931. Pp. go. A list of 
titles appropriate for term papers and theses by graduate students, 
and for practical research projects by men active in administration and 

supervision. 

Smiru, W. R. “Recent Progress in Educational Sociology,” Journal of 
Educational Sociology, III (April, 1930), 450-5. Special mention is 
made of publications, authors, and evidences of a growing insight into 
the values of a socially dominated education. 

*Spaur, W. E. and Swenson, R. J. Methods and Status of Scientific 
Research, Chapter VII. New York: Harper and Brothers, 1930. Select- 
ing and formulating the problem, bibliography, note-taking, con- 
sulting standard reference books, use of special libraries, and gathering 
data outside of libraries. 

SrepHens, W. B. “‘Regimentation’ and Comparable Educational Meas- 
urement,” Junior-Senior High School Clearing House, X (October, 
1935), 114-7. The uses and abuses of new-type devices. 

Strano, Ruth. “Developments in Student Personnel Research,” Teachers 
College Record, XXXV (November, 1933), 120-34. Reviews trends 
with regard to five areas of personnel work—selection of students, 
problems of personality and background, educational guidance, voca- 
tional guidance and part-time employment, and extracurricular 
activities. - 

“Trends in Educational Personnel Research,” Personnel Journal, 
X (October, 1931), 179-88. Analysis of 582 articles published in ten 
magazines during ten years shows trends in teachers’ marks, methods 
of treating data, and in statistical measures. 

Tuersen, W. W. “The Outlook for School Research,” Nation’s Schools, 
XIII (February, 1934), 16-8. A much brighter future is seen than 
policies of retrenchment would indicate. Special stress upon adminis- 
trative research, supplemented by effective publicity, is urged, with 
emphasis on the successes rather than the failures of schools when 
reporting the results of investigations, 

Tuornpike, E. L. “The Need for Fundamental Analysis of Methods of 
Teaching,” Elementary School Journal, XXX (November, 1929), 189- 
9x. Contributions from school experience and laboratory study seem 
to be often adulterated by error or confused by irrelevant factors 
before they operate in terms of methods of teaching; the remedy 
suggested is a more fundamental and thorough analysis of methods of 
teaching rather than in locating blame for past mistakes. 

Toors, H. A. Two Hundred Research Problems for a Five-year Research 
Program into the Collegiate-Entrance Problem. Ohio College Associ- 
ation Bulletin, No. 61. Columbus, Ohio: Ohio State University, 1929. 
Pp. 658-713. f 

Tyrer, R. W. “Evaluation: A Challenge to Progressive Education,” 
Progressive Education, XII (December, 1935), 552-6. Urges formula- 
tion of objectives and determination of the extent to which they are 
realized. 
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Watson, G. B. Experimentation and Measurement in Religious Educa- 
tion. New York: Association Press, 1927. Pp. xii + 296. Prepared for 
students in undergraduate and graduate courses in religious educa- 
tion who are anxious to secure a more adequate understanding of the 
experimental viewpoint and of the necessary technique; includes a 
list of 140 problems. 

Wener, J. J. Visual Aids in Education. Valparaiso University, Valparaiso, 
Ind: Published by author, 1930. Pp. 226 (mimeographed). Includes 
a large number of suggestions for educational research, grouped under 
four major headings, related to: 

1. Psychology of learning and principles of teaching 

Practical utilization of visual aids in the modern curriculum 

Evaluation and improvement of visual aids and mechanical equip- 

ment 
4. Administrative phases of visual instruction in the whole scheme 

of formal education 

Wueetrr, R. H. “Problems of Educational Reconstruction: I, The Issue,” 
Educational Administration and Supervision, XXI (January, 1935); 
23-44. Criticizes education as promoting cultural lag and as prevent- 
ing the growing child from understanding the world in which he 
lives; urges the educator to abandon his atomistic ways of thinking; 
says that education is back on the main track of pre-relativistic, pre- 
organismic thought; and that the problem is to guide the new 
functionalism into sound application and to reconstruct the educa- 
tional program accordingly. 

Wueerter, R. H. “Problems of Educational Reconstruction: II, An Out- 
line of New Objectives,” Educational Administration and Supervision, 
XXI (October, 1935), 491-506. Analyzes twelve objectives. 

Warre, W. A. “The Origin, Growth, and Significance of the Mental- 
Hygiene Movement,” Science, LXXII (July 25, 1930), 77-81. Indi- 
cates some of the outstanding points in the development of the 
mental-hygiene movement, and how the thought of the physician 
has been changed by the introduction of the concept of mental health. 
Predicts that the most significant change which mental hygiene will 
effect is in the concept of values as applied to human beings. 

Warrrorn, W. G. An Introduction to Art Education, Chapter XX. New 
York: D. Appleton and Co., 1929. Deals with research in art educa- 
tion; values of scientific investigation, research and the teacher, the 
broad field of art education, coöperative effort, and the one hundred 
problems in art education. 

*Wuarey, F. L. Methods of Educational Research, Chapter Il. New 
York: D. Appleton and Co., 1931. Deals with the selection of a 
problem for research: origin, location, evaluation, definition, and 
suggested studies. 

Wuey, G. M. “The Contribution of Research to the Organization of 
Secondary Schoels,” School and Society, XXXI (April 19, 1930), 
517-22. Educational workers have been busy with the measurement 
of the educational product, but must now direct thought to the 
improvement of the product and to the development of policies that 
will insure an effective organization for this purpose. 
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Wutey, S. W. “Recent Trends in Character Education,” Religious Educa» 
cation, XXVI (April, 1931), 346-54. Ten trends are reviewed: 


I. 
2. 
3. 
4. 


Se 


The greatly increased concern about character education 

A more critical study of what constitutes character 

The tendency to distinguish between instruction in religion and 
in character training 

Emphasis upon purposeful activity in contrast to dependence 
upon ethical codes 

Emphasis upon the value of small and relatively permanent 
groups 


. Recognition of the issue between standardized programs and 


group-determined programs 


. The issue between teaching a set of virtues and dealing with 


life situations as a whole 


. Interest in parent training 
. Development and use of tests and measurements 
Io, 


Application of the method of scientific research to character 
education 


Wary, M. M. “Recent Social Changes and Educational Research,” 
Journal of Educational Research, XXVII (January, 1934), 321-32. 
Wutson, Lucy L. W. “Three Hundred Years of Education for Girls in 
America,” School and Society, XLII (July 27, 1935), 105-11. An 
historical account of the long struggle which has brought equality 


of 


educational opportunity to girls and women. 


Wooprinc, Maxie N., and Fremine, Cecile W. “Recent Trends in Study,” 
Teachers College Record, XXXVII (October, 1935), 27-49. Covers: 
(1) improvement of the assignment; (2) direction of study in a 
laboratory procedure; (3) acquisition of library skills and techniques, 
and the close relationship of the library and classroom; (4) a new 
evaluation of home study; (5) clearer understanding of the function 


of 


a study hall; (6) a diagnostic and remedial approach to reading as 


a skill, and an appreciation of reading as a recreational activity; 
(7) development of additional study techniques; and (8) discovery 


of 


more effective procedures in specialized subjects. 


* Woopy, Clifford, and Others. “A Symposium on the Effects of Meas- 
urement on Instruction,” Journal of Educational Research, XXVIIL 
(March, 1935), 481-529. 


I. 


5 


All authors recognize the positive potential contribution of meas- 
urement as a means for the improvement of instruction. 


. All seem to agree that the objections to measurement arise from 


faulty techniques of test construction, faulty selection of content, 
and faulty interpretation of results. 


. The authors also seem to agree that the abuses and mijuses of 


measurement present a challenge for improvement in the field 
rather than abandonment of measurement. 


. Most of the participants in this symposium pointed out the need 


for development of instruments of measurement for subtle and 
dynamic qualities such as initiative, coöperation, methods of work, 
methods of thinking, appreciations, attitudes, or emotional effects. 
The contributors implied, if they did not implicitly state, that 
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the various types of measurement must be evaluated in terms of 
the purposes for which they are designed. 

6. The authors seem to imply that values emphasized in the tests 
must change from time to time and must be consistent with the 
essential values of the educational process. 

* Zrepes, Laura. Curriculum Trends: A Preliminary Report and a Chal- 
lenge. Washington: Association for Childhood Education, 1935. Pp 40. 

*“The Activity Movement,” Progressive Education, XI (October, 1934), 
327-59. A symposium based on the “activity” year-book of the 
National Society for the Study of Education, dealing with the activity 
movement in terms of: basic realities, controversial issues relating to 
the activity curriculum, an adverse point of view, and the essentials 
of the movement. 

*The Application of Research Findings to Current Educational Practices. 
Official Report of 1935 Meeting of the American Educational Re- 
search Association. Washington: The Association, 1935. Pp. 273. 

“Educational Books of 1935,” School and Society, XLIII (March 28, 
1936), 409-26. Includes a summary by fields of the number of publi- 
cations, 1927-1935, a total of 5,904 titles for 1926-1035. 

* Evaluating the Public Schools. Washington: National Education Associa- 
tion, 1934. Pp. viii +48. Also Phi Delta Kappan, XVI (February, 
1934), 145-92. Defines educational issues of nation-wide importance 
and cites pertinent evidence, with problems and selected references 
listed. 

* Issues in Secondary Education. Committee on the Orientation of Sec- 
ondary Education, T. H. Briggs, Chairman. Bulletin of the Depart- 
ment of Secondary-School Principals, January, 1936. Chicago: H. V. 
Church, University of Chicago. Pp. 372. See also Journal of the 
National Education Association, XXV (April, 1936), 106; and Nation’s 
Schools, XVII (March, 1936), 32. 

“Problems in Education as Emphasized in Addresses at St. Louis (1936 
meeting of the Department of Superintendence and allied organiza- 
tions),” Phi Delta Kappan, XVIII (March, 1936), 210-8; XVIII 
(April 1936). Reported by W. A. Stumpf. 

*“Problems of Educational Sociology as Seen by the Sociologist,” Journal 
of Educational Sociology, IX (September, 1935), 4-55. A symposium, 
each article having a bearing on either educational theory or prac- 
tice; a unifying element is provided by treating education always as a 
process of social adjustment. 

Significant Changes and Trends in the Teaching of Mathematics through- 
out the World since 1910, Fourth Yearbook of the National Council 
of Teachers of Mathematics, New York: Teachers College, Columbia 
University, 1929. Pp. x + 186. 

Suggestions for Studies and Research in Home-Economics Education: 
Outline of Projects Needing Investigation and Some Basic Considera- 
tions in Relation to Them. Home Economics Series, No. 15. Bulletin 
No. 166. Washington: Federal Board for Vocational Education, 1932. 
Pp. xii + 78. 


CHAPTER III 


THE SURVEY OF RELATED INFORMATION 


Introduction. Although the preceding chapter has been de- 
voted to a detailed analysis of the selection, statement, and 
delimitation of the problem, it is not to be assumed that such 
steps in the process of problem solving are accomplished in- 
dependently of a survey of related information and experience. 
As a matter of fact, the suggestions made in Chapter II and 
in the present chapter are frequently used simultaneously ; 
that is, the investigator may be carrying forward a program of 
careful reading and study in connection with the actual selec- 
tion of his problem, as well as in the later development of the 
plan and procedure formulated. Chapter VI, on the historical 
method, should also prove helpful in making the survey of 
related literature, especially where a time span of some length 
is represented. k 

The present discussion will outline: (1) the purpose and 
contribution of the preliminary survey of related information, 
(2) guides to the various types of educational literature, (3) 
classified lists of important publications with an interest in 
educational problems, and (4) studies of the investigators and 
scientists who produce a factual literature, with special refer- 
ence to education and psychology. 


PURPOSE AND VALUE OF THE PRELIMINARY SURVEY 


Purpose of the survey. The preliminary survey of previous 
studies, discussions, and experience relating to the problem 
under investigation may accomplish five purposes: * (1) show 


. 1For a list phrased differently but with similar emphasis, see: Carter Alexander, How 
to Locate Educational Information and Data, pp. 6-8. New York: Bureau of Publications, 
Teachers College, Columbia University, 1935. 
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whether the evidence already available solves the problem 
adequately without further investigation; (2) provide ideas, 
theories, explanations, or hypotheses valuable in formulating 
the problem; (3) suggest methods of research appropriate to 
the solution of the problem; (4) locate comparative data use- 
ful in the interpretation of results; and (5) contribute to the 
general scholarship of the investigator. Since other chapters 
deal with the formulation and testing of hypotheses, methods 
of research for collecting data, and interpretation of results, 
the contribution of the survey of related information to these 
procedures will not be discussed at this time; it plays an im- 
portant part in each of these steps. 

Avoidance of unnecessary duplication. To prevent point- 
less repetition, the investigator, whether in the field or the 
graduate school, who has chosen a definite problem for study 
must conduct a thorough canvass of the materials already 
available in the given field. As a matter of fact, in all prob- 
ability, he has made at least a partial bibliographical survey 
in the selection of a problem. Only through painstaking an- 
alysis of the available educational literature is the research 
worker able to determine whether his chosen problem or some 
phase of it has been previously investigated. If this preliminary 
step is neglected, he may be attacking a problem which has 
been satisfactorily worked out. A university teacher of Italian, 
a Ph.D. candidate, had worked for a year on the contribution 
of a certain Italian author to the literature of his time. Imagine 
the disappointment and chagrin of this teacher to find acci- 
dentally on the library shelves two dust-covered volumes on 
the same subject, written by a recognized scholar in a much 
more thorough-going manner than he (the candidate for a 
degree) could hope to achieve. Undoubtedly, the preliminary 
canvass of the field had been far from complete. For that 
matter, when as great a scientist as Pasteur found that the 
bacillus which manufactures rancid-butter acid worked with- 
out air, he announced it as the first example of little animals 
living without air; the literature would have shown him that it 
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was in reality the third example and have saved his time and 
dignity.” 

Avoidance of duplication in the case of the doctoral disserta- 
tion has been especially stressed by Alexander who considers 
this one of the most serious types of duplication, and believes 
that the degree cannot be granted if the thesis overlaps to a 
large extent any previously published work.' Unfortunately 
there are some instances where the candidate has not been in- 
tellectually honest and has appropriated large portions of the 
work of another without acknowledging his indebtedness, al- 
though duplication of effort is generally caused by ignorance 
of the earlier work. 

A Doctor’s candidate who had spent nearly a year in studying 
the problem of certification of teachers in a certain state was 
very much surprised in 1935 at the appearance of a printed 
dissertation on the same subject (and state) from another uni- 
versity. He could have found the latter thesis reported as under 
way in the January, 1933, number of the Journal of Educational 
Research, at least a year before he began work on the problem. 

The question of duplication is sometimes raised when two 
students wish to work out a joint graduate thesis, a procedure 
which is not encouraged as a general rule. There are on record 
a few instances in which the problem has seemed so large, and 
at the same time so urgent, that two workers were permitted 
to submit a joint manuscript, for example, duplication of cur- 
riculum content in the various departments and colleges of a 
large state university. As a rule, a better measure of the 
initiative and performance of each student is provided by 
breaking up a large problem into two parts, if such division is 
possible. For example, a study of curriculum reorganization in 
the secondary schools of a state proved too comprehensive for 
a single Master’s thesis, and the project was subdivided into 
two Masters’ problems, one concerned with the junior high 
schools and the other with the senior high schools. The two 


2T. A. Boyd, Research, pp. 74-5. New York: D. Appleton-Century Co., 1935. 
3 Carter Alexander, op. cit., pp. 227-8. 
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students were able to codperate in a number of ways—in 
traveling, visiting school systems, examination of records, dis- 
cussion of data, etc.—and the procedure had value in the trend 
toward coérdination in research. 

Full-time students and teachers in service registered in uni- 
versity classes sometimes propose as subjects for term papers, 
and even theses, the preparation of comprehensive summaries 
of factual studies in such fields as reading and arithmetic, 
seemingly unaware of the excellent work of Gray * and Bus- 
well® respectively and of the annual supplements in the 
Elementary School Journal and Journal of Educational Re- 
search. Of course the height of the ridiculous is represented in 
the case of the student described in Chapter II who laid before 
Thorndike a manuscript, basing his hope of acceptance on the 
originality represented, because he had read nothing whatever 
on the subject treated.* In Chapter II is an account of another 
student, who proposed that he correlate intelligence and 
achievement, seemingly unaware that this has been done many 
hundreds of times.’ 

Mistaken interpretations of meaning of duplication. Young 
and inexperienced workers sometimes take Crawford’s advice 
to “plan your investigation before reading what others have 
done” too literally; in reality he simply wishes the worker to 
make his plans as complete as possible, before canvassing in 
detail the related studies, in order that originality and initia. 
tive may have an opportunity to function without undue 
influence operating in the form of a minute analysis of pre- 
viously employed techniques.* Of course the original plan of 
the investigation may be refined materially in the light of an 

4W. S. Gray, Summary of Investigations Relating to Reading. Supplementary Educa- 
tional Monographs, No. 28. Chicago: Department of Education, University of Chicago, 
1925. Pp. viii + 276. 

5G. T. Buswell and C. H. Judd, Summary of Educational Investigations Relating to 
Arithmetic. Supplementary Educational Monographs, No. 27. Chicago: Department of 
Education, University of Chicago, 1925. Pp. viii + 212. 
ee McCall, How to Experiment in Education, p. 8. New York: The Macmillan 


7 “The Editor Turns Professor,” Educational Research Bulletin, VI (September 14, 


4927), 252-53. 
8C. C. Crawford, Methods of Study, p. 162, Moscow, Idaho: C. C. Crawford, 1926. 
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exhaustive examination of the procedures and conclusions of 
related studies, which is a very desirable outcome of the bib- 
liographical survey. 

It happens occasionally that two workers, each without 
knowledge of the other, conduct similar studies simultaneously 
and present almost identical results at the same time. 
Without question both make an important joint contribution 
to knowledge, for example, the James-Lange theory of the 
emotions, the Lancaster-Bell system of monitorial instruction, 
the contribution of the Wright brothers and Langley to 
heavier-than-air flying, and the discovery of anesthetics. 

Sometimes the beginner in educational research mistakenly 
assumes that, because a problem has been studied under one 
set of conditions, the issue is settled for all other circumstances. 
A high-school principal in West Virginia was studying as a 
Master’s problem the all-year school for mining communities. 
He became discouraged without cause on finding that the all- 
year school had been studied in connection with a single school 
in metropolitan New York City. For that matter, in certain 
instances, studies need to be repeated under identical condi- 
tions in order to secure an adequate check on the results of 
the first investigation. The normative-survey type of investiga- 
tion, which deals with current conditions, needs to be repeated 
at fairly regular time intervals in order to keep in touch with 
changes and developments. 

Survey of related information as a part of the report. The 
material secured from the preliminary survey of the field 
usually makes up one of the early chapters of the research 
report and serves to orient the reader, as well as the author. 
Later, in interpreting results, significant comparisons may be 
made with the conclusions found in related studies. Obviously, 
in the supervision of research and in the acceptance of theses 
or investigational reports, it is as important for the university 
professor, research director, and public-school supervisor or 
administrator to know the field involved as it is for the gradu- 
ate student and teacher. 


tl 
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Other values of familiarity with educational literature. 
One avenue to scholarship is through accurate knowledge of 
‘the evidence and literature in one’s area of interest or activity, 
whether in a higher institution, research organization, or field 
position. Most graduate schools properly emphasize this aspect 
of advanced training in evaluating the competence of candi- 
dates for degrees? and can cite examples of certain students 
who have passed their examinations with flying colors, because 
of such competence. On the other hand, some workers have 
been delayed in passing their graduate examinations, or at 
least considerably embarrassed during the ordeal, owing to a 
limited knowledge of educational literature. This condition 
may result from one or more reasons, such as, lapse of time 
since graduate courses and residence work were completed, 
preoccupation with non-professional activities, distribution of 
work oyer a large number of scattered summer sessions, or in- 
tensive specialization in some limited area of education. It is 
true that unusual competence in a single subdivision of knowl- 
edge such as educational psychology and an exceptional thesis 
may offset in part, in the minds of faculty members, the 
shortcomings of the candidate in certain other areas, although 
the awarding of the Doctor’s degree in education is usually 
based on broad knowledge of the field as a whole. It is to be 
hoped that similar ideals of scholarship will be maintained iv 
the years of field service following the completion of a program 
of graduate training. Certainly the solution of many field prob- 
lems must depend to a considerable extent on the research 
attitude and investigational knowledge of this nucleus of edu- 
cational workers. The foregoing statements are not meant to 
imply that the graduate school is the only medium of training 
for problem solving. Individual reading and study and ade- 
quate supervisory contacts may, and should, contribute valu- 
able assistance in problem solving. 

It,has been properly emphasized that, as a consumer of 


9 Carter V, Good, How to Do Research in Education, Chapter II. Baltimore: Warwick 
and York, 1929. 
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research, the field worker is obligated to examine carefully all 
the evidence on his problem and to evaluate it critically before 
deciding which of the conflicting results found in the literature 
should be adopted and applied to practice.° Another chapter 
(XIV), however, deals at some length with the problem of 
evaluating educational research.” 

Extent of information possessed by educational workers. 
However, the evidence is all too clear that not only students 
in training and field workers in service but even authors are 
ignorant of important evidence and literature in the field in- 
volved. An editorial writer took to task an officer of the 
(then) Federal Board for Vocational Education who used an- 
tiquated statistics in an address. The speaker had stated that 
only 34 per cent of fifth-grade pupils entered high school, 
whereas, according to the editorial writer, 53 per cent of the 
young people of high-school age were in secondary schools at 
that time.’* 

Even concerning a comparatively recent survey of national 
scope and significance, it has been reported that the findings are 
not known and used to the extent expected and desired. 

Some factual evidence concerning the educational informa- 
tion possessed by school workers is available. In a monograph 
edited by Pressey, D. A. Worcester reveals wide gaps in the 
background of students in education concerning educational 
and scientific organizations, honorary professional organiza- 
tions, educational journals, universities prominent in educa- 
tion, educational leaders, and requirements for graduate 
degrees."* The group investigated consisted of 219 students 
attending summer session in two well-known colleges of educa- 
tion. Of this group 73 per cent had teaching experience, 47 


10 W. A. Brownell, “The Field Worker as a Consumer and Producer of Research,” 
Journal of Educational Research, XXIX (October, 1935), 144-8. 

11 Carter V. Good, op. cit., Chapter II. 

12 “Antiquated Statistics,” Elementary School Journal, XXX (September, 1929), 4. 

13G. E. Carrothers, “Survey of Secondary Education,” School and Society, XLI 
(May 4, 1935), 607-10. A questionnaire canvass of the extent to which the survey has 
been used in Michigan; indicates the need for wider use of these monographs. 

14S. L. Pressey, L. C. Pressey, and Others, Research Adventures in University Tesch- 
we, pp. 63-5. Bloomington, Ill.: Public School Publishing Co., 1927. 
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per cent having taught three years or more; 62 per cent were 
in or above the third year of their professional course. When 
asked to list educational and scientific organizations, 20 per 
cent failed to name any, 8 per cent named only local bodies, 
and no national organization other than the National Educa- 
tion Association (54 per cent) was mentioned by more than 
5 per cent of the students (except the National Geographic 
Society, listed by 9 per cent). No honorary professional organi- 
zation was known by more than 12 per cent. It was faund that 
7 per cent could not name an educational journal; 9 per cent 
could list only local or state periodicals; and no single journal 
was mentioned by more than 24 per cent. When asked to name 
universities prominent in education, more than one-half of the 
students listed only two institutions. No single leader in 
education was mentioned by more than 46 per cent, and the 
field of specialization of the worker named was in most cases 
unknown. 

Ashbaugh tested the information possessed by forty-eight 
students concerning educational workers through a matching 
exercise which included the names of thirty well-known (at 
that time) educators and a list of twenty statements." Eack 
statement was to be matched with the appropriate name. 

Although some of the offices represented had only temporary, 
local, or state-wide significance or prominence, they should 
have been familiar at the time of the investigation to students 
in the state involved (Ohio). Correct responses were given on 
38 per cent of the items. Only 63 per cent of the total number 
of items were attempted. Seven different people were desig- 
nated as the president of the N. E. A., as the dean of Teachers 
College of Columbia University, and as the president of the 
University of Chicago; six different persons were named as 
the editor of the Educational Review. 

Ashbaugh also reports the information possessed by sixty-six 
graduate students concerning the positions held by fifty writers 


15E, J. Ashbaugh, “Should School Administrators Know Something About People 
Prominent in Education?” Educational Research Bulletin, VI (Octoher 12, 1927), 291-3. 
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who had contributed three or more articles to educational 
journals of national scope during the years immediately pre- 
ceding the investigation. Of the total 3,300 possible responses 
(fifty names and sixty-six students) only 615 were attempted, 
and 420 were correct. Only 18.6 per cent of the total items 
were attempted, and but 68.3 per cent of those tried were 
answered correctly. 

Such facts as those’ summarized in the preceding paragraphs 
suggest that the present status of professional information is 
in many instances at a comparatively low level and that pro- 
fessionalization of the rank and file of field workers, in terms 
of an adequate working knowledge of educationai literature, 
must be a gradual development. However, it seems reasonable 
to expect graduate students in education, school administrators, 
supervisors, department heads, most secondary-school teachers, 
and the best trained elementary workers to be fairly well 
acquainted with the service of leaders in the field and with sig- 
nificant contributors to educational literature. Ordinarily such 
groups of workers have completed an undergraduate program 
of training with a prescribed number of courses in education, 
and in many cases they have pursued graduate courses in the 
area of specialization represented. The assumption is that they 
have read more or less regularly one or more of the educational 
periodicals. Therefore, the natural expectancy of the professor 
of education, the progressive school administrator, and the 
supervisor is that graduate students and field workers will 
possess or acquire information as needed, concerning and from 
the various types of educational literature discussed in suc- 
ceeding portions of this chapter. Amplifications of the point of 
view expressed in this and preceding paragraphs may be found 
in numerous current discussions.** 


16 Carter Alexander, “Selecting Magazines for Teachers Starved for Professional Vita- 
mins,” School Executives Magazine, LII (April, 1933), 268-70. 

F. E. Bolton, “The Professional Library in the School System,” School and Society, 
XXXIX (January 13, 1934), 59-63. 

J. H. Newlon, “The Principal’s Professional Library,” School Executives Magazine, 
LIII (October, 1933), 50-1. 

J. R. Shannon, “Directing Professional Reading as a Device in Supervision of Instruc- 
tion,” Educational Method, XIIE (March, 1934), 308-11. 
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Need for a knowledge of library procedure. The gaps in 
professional information revealed in the preceding paragraphs 
suggest the desirability of systematic reading and study on the 
part of school workers in service and in training, at least in the 
major field of interest and specialization represented. The 
library knowledge and skill needed by professionally alert field 
workers are listed by Alexander under eighty-three headings, 
from which five are quoted here to indicate the helpfulness of 
the entire list and the accompanying discussion. Such a check- 
list may be used by the educational worker as a self-survey of 
his knowledge and deficiency." 


1. Where can I find the significant facts about a community or 
an institution in which I would like to work or apply for a position? 

4. Do I know when and how to skim in library reading? 

7. Is there in existence a guide to the professional literature of 
my field? If so, where is it and how may I secure a copy? If there is 
no such guide, what ought to go into one so that I may keep my 
eyes open and in the course of several years gradually build one up 
for myself? 

9. Which encyclopedia is best for use in my major field? 

10. Which are the most useful three reference books for my 
major field? 


The same writer *® outlines in some detail the necessity for 
planning intelligently the search for library materials, mastery 
of the various types of reading skills needed, use of appropriate 
headings for the search, and knowledge of sound principles of 
note taking; and reproduces an unusually complete form of 
bibliography card.*° 

The Voegelein list of subject headings (alphabetically ar- 
ranged),?° the Education Index, and the card catalogue of the 


17 Carter Alexander, How to Locate Educational Information and Data, op. cit., pp. 
13-8, 

18 Ibid., pp. 21-27, 28-30, 65-79, 80-90. 

19 Carter Alexander, Alexander Universal Bibliography Card. New York: Bureau of 
Publications, Teachers College, Columbia University. 1934. 

For a comprehensive loose-leaf system for the collection and use of references anti 
other data, see the Visible Index Research System. Distributed by Crossman and Clayton, 
327 Broadway, New York City, or Paul V. West, School of Education, New York 
University, New York. 

20 L. Belle Voegelein, List of Educational Subject Headings. Columbus, Ohio: Ohio 
State University Press, 1928- 
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library are useful in formulating a list of subject headings for 
the bibliographical survey. A new list of subject headings (a 
classified organization with an alphabetical index) which in- 
cludes definitions of many of the more controversial and less 
well-known topics in education is now in preparation. 

Although the topic mentioned as an example in the subse- 
quent discussion of educational guides is “teacher personnel,” 
other subjects, such as “adaptation of the instructional pro- 
gram to varying needs,” would be canvassed in a similar man- 
ner, of course first ascertaining appropriate subject headings. 
In the case of the latter topic such headings as individual 
differences, individualized instruction, interests, unit organiza- 
tion, curriculum reorganization, directed or supervised study, 
ability grouping, remedial teaching, and many others would be 
involved. 

Obviously, it is essential that a systematic record be kept ot 
worth-while conclusions derived from such reading and an- 
alysis; otherwise, valuable references and data cannot be re- 
called when desired. Probably many incomplete references and 
some haziness in papers, reports, and even in published 
materials are caused by failure to copy complete bibliograph- 
ical information and to prepare an adequate annotation at the 
time of the original reading. Cards or slips of paper have beer 
found better than large sheets of paper for filing such infor- 
mation. Detailed examples of bibliographical form may be 
found in a later chapter (XIIT) dealing with the reporting of 
research, and note-taking is discussed in Chapter VI. 


SOURCES OF INFORMATION AND TYPES 
OF EDUCATIONAL LITERATURE 


The major types of educational literature from which as- 
sistance and information may be secured are listed in terms 
of the kind of thinking or research procedure involved: (1) 
statistical or experimental studies with emphasis on objective 
methods and results expressed mathematically ; (2) theoretical, 
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speculative, or philosophical discussions with emphasis on the 
methods of logic and rationalization; and (3) historical inves- 
tigations. In later chapters each of these types will be given 
reasonably full consideration. In view of this fact, and espe- 
cially because library aids and guides are not classified along 
the foregoing lines, the remainder of the present chapter must 
be organized to facilitate ready use of the available informa- 
tion. Therefore, the workable classification has been adopted 
in terms of the form in which the material appears: (1) 
periodical literature (found in journals); (2) books, mono- 
graphs, year-books, and bulletins; (3) graduate theses, which 
usually are unpublished except for a limited number of Doc- 
tors’ dissertations; and (4) certain miscellaneous sources of 
information. Before entering upon a detailed discussion of each 
of these specialized types of educational literature, a brief but 
systematic plan should be given for making a reasonably com- 
plete and rapid initial survey of the entire field of literature 
dealing with the problem under investigation. This is done 
under the heading of educational “guides.” 

Guides to educational literature. Really, in spite of the 
greatly increased mass of educational literature, it is so much 
simpler to-day than tifteen or even ten years ago, to find 
quickly the previous studies germane to a given problem, that 
further encouragement may well be given to the workers who 
have prepared such aids as those mentioned below. This fact 
makes it all the more inexcusable when investigators have 
been ignorant or careless in analyzing earlier studies in the 
area represented. It is possible to note the extent to which 
Libliographical aids and library guides have increased by com- 
paring earlier discussions of this problem with the range of 
information presented in this chapter.** If subsequent para- 


21E. J. Ashbaugh, “A Bibliography of Research Articles Published in 1924,” Journal 
of Educational Research, XI (May, 1925), 368-79. 

Arthur Edward Larkin, “The Historical Development of the Current Indexing of 
_ Published Educational Information.” Unpublished Master’s thesis, University of Notre 
Dame, 1932. 

Harold O. Rugg, “How to Keep in Touch with the Quantitative Literature of Edu- 
cation,” Elementary School Journal, XVIII (December, 1917), 301-10. 
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graphs seem elementary to the advanced university student of 
educational problems, it will be conceded that they offer useful 
suggestions to most field workers and to beginners in educa- 
tional research who are vften literally “swamped” in attempt- 
ing to canvass a pertinent part of the great volume of litera- 
ture now available. 

General guides. Let us assume that the area of interest is 
“teacher personnel.” For purposes of orientation it is probably 
best to read the professional textbooks or reference works on 
this subject discussed later in this chapter. Books on school 
administration and supervision usually include chapters on 
the teaching staff which provide additional selected references 
dealing with the topic under investigation. 

The Education Index, which began publication in 1920, 
could well be examined next, since it is the most important 
educational guide in existence. Its purpose is to index all edu- 
cational books in the United States, except elementary and 
high-school textbooks; all articles in a large number of educa- 
tional periodicals; other educational articles appearing in 
certain non-educational journals; and a large number of mono- 
graphs, bulletins, reports, book reviews, courses of study, state 
documents, etc., of interest to educational workers.2* A good 


» Statistical Methods Applied to Education, Chapter II. Boston: Houghton 
Mifflin Co., 1917, 

W. W. Theisen, “Recent Progress in Educational Research,” Journal of Educational 
Research, VIII (November, 1923), 301-14. Summarizes the chief contributions of ze- 
search to instruction and administration, and suggests other investigations that are 
needed in these fields. 

M. R. Trabue, “Educational Research in 1925,” Journal of Educational Research, 
XIII (May, 1926), 336-44. A summary of the literature on educational research, articles 
in periodicals, books, and booklets, published during 1924 and 1925. Mentions a number 
of investigations as having special merit. 

Clifford Woody, “A Survey of Educational Research in 1923,” Journal of Educationat 
Research, IX (May, 1924), 357-81. A review of the important researches reported in 
1923 in various fields, including administration, instruction, educational and mental 
tests, statistical methods, experimental studies, and miscellaneous. 

22 Because of the fundamental value of the Education Index in referencing important 
educational literature, many educators will undoubtedly desire to subscribe to this service 
personally. The annual cumulative cloth-bound volume may be secured for approxi- 
mately the price of an ordinary educational periodical, and the entire indexing service, 
including paper-bound monthly publications (which make the information in the annual 
Volumes available each month right after the educational periodicals have been pub- 
lished) may be secured for twice this amount, Write the H. W. Wilson Company, 950 
University Avenue. New York City, for particulaxs, 
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statement of the organization and use of the Education Index 
is available elsewhere.** 

A logical starting point, in terms of a variety of research 
references, is to examine the Review of Educational Research, 
which began publication in 1931 and covers some fifteen sub- 
divisions of education within a three-year cycle. Fortunately 
for our investigator, the June, 1934, number of this digest of 
research includes 640 references on teacher personnel and re- 
views of the studies relating to thirteen major problems in 
this field. The April, 1931, issue includes a bibliography of 
458 items relating to teacher personnel. Presumably, in 1937 
another number of the Review of Educational Research will 
deal with this same problem. The canvass may be extended by 
examining the annual bibliographies of research published by 
the United States Office of Education, the classified and 
selected lists of references published monthly in the School 
Review and Elementary School Journal, Monroe’s Ten Years 
of Educational Research (1918-1927),?* and Locating Educa- 
tional Information in Published Sources, Monroe and Shores’ 
A Catalogue of Bibliographies and Summaries in Education to 
June 30, 1935,°° Alexander’s Educational Research," and the 
Research Bulletin of the N.E.A. 

The Psychological Bulletin publishes frequent bibliograph- 
ical summaries of research in the various subdivisions of psy- 
chology, including educational psychology. The Educational 
Research Bulletin of the Catholic University of America has is- 
sued reviews and bibliographies covering a variety of fields— 


23 Carter Alexander, How to Locate Educational Information and Data, op. cit., pp 
95-100. 

24 W. S. Monroe and Others, Ten Years of Educational Research, 1918-1927. Uni- 
versity of Illinois Bulletin, Vol. XXV, No. 51. Bureau of Educational Research Bulletin, 
No. 42. Urbana, Ill.: University of Illinois, 1928. Pp. 368. 

25 W. S, Monroe and Others, Locating Educational Information in Published Sources. 
University of Illinois Bulletin, Vol. XXVII, No. 45. Bureau of Educational Research 
Bulletin, No. 50. Urbana, Ill.: University of Illinois, 1930. Pp. 142. 

26W. S. Monroe and Louis Shores, A Catalogue of Bibliographies and Summaries in 
Education to June 30, 1935. New York: The H. W. Wilson Co., 1936. To be continued 
in the Education Index. - 

27 Carter Alexander, Educational Research: Suggestions and Sources of Data with 
Specific Reference to Administration. Third Edition, Revised. New York: Bureau of 
Publications, Teachers College, Columbia University, 1931. Pp. 11s. 
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the constancy of the intelligence quotient, reading tests, spelling, 
child development, ability grouping, reading interests, etc. A 
new journal, Educational Abstracts, established in January, 
1936, will present concise abstracts from books, journals, and 
monographs in both American and foreign literature five times 
a year. The major topics covered by the Review of Educational 
Research, School Review, Elementary School Journal, and 
Psychological Bulletin are given in the following lists. 


Torics Coverep IN THE “Review or EDUCATIONAL RESEARCH” 


“The Curriculum,” I (January, 1931), 1-64. 

“Teacher Personnel,” I (April, 1931), 65-160. 

“School Organization,” I (June, 1931), 161-244. 

“Special Methods in Elementary School,’ I (October, 1931), 245-324. 

“Psychology of School Subjects,” I (December, 1931), 325-450. 

“Special Methods on High-School Level,” II (February, 1932), 1-94. 

“Finance and Business Administration,” II (April, 1932), 95-182. 

“Tests of Personality and Character,” II (June, 1932), 183-270. 

“Tests of Intelligence and Aptitude,” II (October, 1932), 271-342. 

“School Buildings, Grounds, Equipment, Apparatus, and Supplies,” II 
(December, 1932), 343-441. 

"Educational Tests and Their Uses,” III (February, 1933), 1-80. 

“Mental and Physical Development,” III (April, 1933), 81-182. 

“Pupil Personnel, Guidance, and Counseling,” III (June, 1933), 183- 
278. 

“Psychology of Learning, General Methods of Teaching, and Super- 
vision,” III (October, 1933), 279-368. 

“The Legal Basis of Education,” III (December, 1933), 369-482. 

“Methods and Techniques of Educational Research,” IV (February, 
1934), 1-119. 

“The Curriculum,” IV (April, 1934), 121-252. 

“Teacher Personnel,” IV (June, 1034), 253-352. 

“School Organization,” IV (October, 1934), 353-444. 

“Psychology and Methods in the High School and College,” IV 
(December, 1034), 445-563. 

“Special Methods and Psychology of the Elementary-School Subjects,” 
V (February, 1935), 1-120. 

“Finance and Business Administration,” V (April, 1935), 121-184. 

“Psychological Tests,” V (June, 1035), 185-331. 

28 The monthly selected references in the School Review and Elementary School Journal 
cumulate annually and are bound under one cover, for example, Selected References in 


Edycation, 1934. Supplementary Educational Monographs, No. 42. Chicago: Departmenr 
ot Education, University of Chicago, January, 1935. Pp. x + 190. 


THE SURVEY OF RELATED INFORMATION 119 


“The School Plant,” V (October, 1035), 333-434- 

“Educational Tests and Their Uses,” V (December, 1935), 441-520. 

“Mental and Physical Development,” VI (February, 1936), 1-152. 

“Student Accounting, Personnel, and Guidance,” VI (April, 1936). 

“Psychology of Learning, General Methods, and Supervision,” VI 
(June, 1936). 

“History of Education and Comparative Education,” VI (October, 
1936). 

“Mental Hygiene,” VI (December, 1936). 

“Educational Sociology,” VII (February, 1937). 


Topics COVERED IN THE MONTHLY SELECTED REFERENCES IN THE 
“Scnoot Review” 


Guidance. aena adaa eaaa a EE URES 
Organization of Secondary Education. . . - . + October 
Administration of Secondary Education . . . . > November 
Higher Education. FATE «s+ + + © t+ 4 December 
Secondary-School Instruction: I. >. + + + + + January 
Secondary-School Instruction: II. - - + + + + February 
Secondary-School Instruction: WI. . - +. + = + March 
The Extra-Curriculum =... +. +» + - e + April 
Educational Psychology > . > + + + + + = + May 
Statistics and the Theory of Test Construction. . . June 


Topics Covexep IN THE MONTHLY SELECTED REFERENCES IN THE 
“EpemMenTARY SCHOOL JOURNAL” 


Elementary-School Instruction: I, Curriculum, Methods 
of Teaching and Study, and Supervision . . . - September 

Elementary-School Instruction: II, The Subject Fields . October 

Elementary-School Instruction: III, The Subject Fields 


(Continued) oaiae gaa Wists November 
Teacher Education - si. aan + + © n iaaa December 
Public-School Administration : gnc oe ese ES Naa ETENN 
Public-School Administration: II . . . - . - . February 
Preschool and Parental Education . . - + + + + March 
Kindergarten-Primary Education . - - + + + + April 
Literature of Exceptional Children. - - + + + + May 
Foreign Education . . Tart anode teat este (fet 


SELECTED Topics REVIEWED IN THE “PSYCHOLOGICAL BULLETIN” 


Intelligence Tests 
Educational Tests 
Character and Personality Tests 
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Sex Differences 

Learning Theories 

Learning in Infant and Preschool Children 
Imagination 

Social Psychology of Everyday Life 
Introversion-Extroversion 

Personality Adjustments of Only Children 
Interferences in Reading 

Influence of Disease on Motor Development During Childhood 
Experimental Study of Reading 

Childhood Play 

Learning and Retention of Verbal Material 
Law of Effect As a Principle of Learning 


Periodical guides. It is possible to make the canvass more 
detailed and complete by use of certain bibliographical and 
library aids organized in terms of the type of publication rep- 
resented. The periodical literature is well covered in certain 
of the sources mentioned in the preceding paragraph, especially 
the Education Index, as far as a complete classification and 
listing of titles is concerned. Other periodical indexes and 
abstracts are: Readers’ Guide to Periodical Literature, Inter- 
national Index to Periodicals, Teachers Journal and Abstract 
(suspended publication in 1932), Educational A bstracts, Edu- 
cation Digest, Loyola Digest, Psychological Abstracts, Psycho- 
logical Index, Occupational Index, and Social Science Abstracts 
(suspended publication in 1932). It will be noted from the 
titles of certain of these guides that they provide an abstract 
or digest service, while other aids are devoted to the task of 
indexing periodical literature. In subject-matter fields other 
than education and psychology may be found similar library 
guides, as listed in the chapter bibliography. 

Helpful tables, showing the years covered by various 
periodical indexes and the chronological order in which indexes 
should be consulted for certain types of information, are avail- 
able.?? 


29 Carter Alexander, How to Locate Educational Information and Data, op. cit., Pp. 
110-1, A 

Eleanor M. Witmer and Margaret C. Miller, “Guides to Educational Literature jn 
Periodicals,” Teachers College Record, XXXIU (May, 1932), 724-5, 
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Guides to books and monographs. Special additional assist- 
ance is available for canvassing non-periodical literature. For 
a number of years School and Society has published annually 
a complete classified list of educational books, monographs, 
year-books, and bulletins of the preceding year with the so- 
called “sixty most important” books of the year indicated. 
This same list of sixty books appears annually in a spring 
number of the Journal of the National Education Association, 
which also publishes an annual list of forty “notable” Amer- 
ican books covering the entire range of intellectual endeavor. 
The complete lists in School and Society include 813 titles in 
1933, 765 in 1934, and $23 in 1935. The numbers of School and 
Society which should be consulted are: 


XXV (January 1, 1927), 18-23; for 1926. 
XXVI (December 31, 1927), 823-35; for 1927. 
XXVIII (December 29, 1928), 801-15; for 1928. 
XXX (December 28, 1929), 857-73; for 1929. 
XXXIII (January 24, 1931), 101-21; for 1930. 
XXXV (March ro, 1932), 393-410; for 1931. 
XXXVII (March 25, 1933), 391-408; for 1932. 
XXXIX (March 24. 1934), 374-92; for 1033. 
XLI (March 30, 1935), 432-48; for 1934. 
XLIII (March 28, 1936), 409-26; for 1935. 


Books and monographs may also be located by author, title, 
or subject in the card indexes of the library, the United States 
Catalogue, Supplements to the United States Catalogue, and 
the Cumulative Book Index, which give complete information 
concerning publisher, number of pages, price, etc. The Pub- 
lishers’ Weekly may be used when it is essential to secure in- 
formation concerning books which have appeared since the last 
number of the Cumulative Book Index. The service of the 
Education Index in listing books has already been described. 
Other guides to books and literature are given in the chapter 
bibliography. The Book Review Digest is of value in providing 
brief quotations from book reviews previously published in 
periodicals. This service aids in the evaluation of books 
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Guides to graduate theses. The discovery and examination of 
graduate theses involve more difficulty than the materials 
mentioned above, since most Masters’ and many Doctors’ 
studies are unpublished. However, reasonably good aids are 
available even for canvassing these unpublished materials. 
Over the period, 1917-1927, the Bureau of Educational Re- 
search of the University of Illinois prepared mimeographed 
annual or biennial lists of graduate theses in education. 
Monroe’s Ten Years of Educational Research (1918-1927) *° 
Includes the titles of Doctors’ dissertations in education over 
the period covered. Since that time the annual bibliographies 
of educational research issued by the United States Office of 
Education have been devoted very largely to an annotated list- 
ing of thesis titles. The bulletins of the Office of Education 
which should be consulted for titles of Masters’ and Doctors’ 
theses in education are: 


Bulletin No. 22, 1928 for 1926-27. 
Bulletin No. 36, 1929 for 1927-28. 
Bulletin No. 23, 1930 for 1928-29. 
Bulletin No. 13, 1931 for 1929-30. 
Bulletin No. 16, 1932 for 1930-31. 
Bulletin No. 6, 1933 for 1931-32. 
Bulletin No. 7, 1934 for 1932-33. 
Bulletin No. 5, 1935 for 1033-34. 


The Office of Education also has on file a number of these 
studies for lending purposes, 1900 in 1935. At intervals the 
Office publishes a list of theses which have become available 
for loan since the last previously published list. Interlibrary 
loans of theses may also be made between educational institu- 
tions.*? Within recent years many graduate institutions have 
published volumes of abstracts of completed theses, and this 

80 W. S, Monroe and Others, Ten Years of Educational Research, 1918-1927, op. cit. 

81 Recent Theses in Education, Office of Education Pamphlet No. 26, 1931. Washing- 
ton: Office of Education, 1932. Pp. 41. 

Ruth A, Gray, Doctors’ Theses in Education: A List of 797 Theses Deposited with 
the Office of Education and Available for Loan. Pamphlet No. 60. Washington: Office 
of Education, 1935. Pp. 70. 


82 Carter Alexander, How to Locate Educational Information and Data, op. cit., pp. 
ts2-9, A discussion of the procedure involved in interlibrary loans. 
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information is available in a later section of this chapter, in 
the chapter bibliography, and elsewhere.°° 

Later in this chapter, reference is made to lists of doctoral 
dissertations published by the Library of Congress and by the 
H. W. Wilson Co. 

Lists of theses may be located in the Education Index under 
the topics, “dissertations—academic” and “degrees—academic, 
Doctor’s, and Master’s.” 

The Journal of Educational Research prints annually each 
January a list of Doctors’ studies under way in education. 

Other bibliographical and summary guides. In addition to 
guides to educational literature previously mentioned, other 
specialized aids (bibliographies or summaries of research) 
should be mentioned. The classified monthly lists of references 
in the School Review and Elementary School Journal, and the 
summarizing work of the Review of Educational Research have 
already been described, as has the Psychological Bulletin 
which publishes critical summaries of research dealing with a 
wide range of psychological problems. Within recent years the 
literature in the field of psychology has been summarized in an 
admirable manner in the handbooks of psychology, edited by 
Carl Murchison and published by Clark University Press. The 
titles of the bibliographical reviews mentioned above may be 
found annually in the September number of the Journal of 
Educational Research.** Certain journals, such as the Research 
Quarterly of the American Physical Education Association, 
Modern Language Journal, Religious Education, and Science 
Education, have been publishing annual bibliographies in the 
fields represented for several years. To illustrate the valuable 
bibliographical and summary aids now available in education, 
only a few of the more comprehensive attempts can be 
enumerated in the chapter bibliography: 


88 Clara E. Derring, “Lists and Abstracts of Masters’ and Doctors’ Dissertations in 
Education,” Teachers College Record, XXXIV (March, 1933), 490-502. 

84 For example, Carter V. Good, “Selected Bibliography on the Methodology of Edu- 
cational Research and Related Problems,” Journal of Educational Research, XXIX 
(October, 1935), 156-70. 
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Higher education and teacher training Good, Betts 


Personnel development and guidance Strang 


Junior college 


Buildings, grounds, and equipment A 


School finance . 

Research methodology 5 
Behavior development in infants . 
Heredity and environment 
Mental tests and rating scales . 
Mental tests and heredity 

Social science X 

English and rados 

Arithmetic 


Science i 
Foreign language 


Athletics 

Commercial education! 
Religious education 
Leadership 

Examinations 

National systems af education 


Eells 

Smith and Noffsinge: 

Alexander and Covert 

Barr and Rudisill, Good 

Dewey 

Schwesinger | 

Hildreth 

Schieffelin and Schwesinger 

Bye, Ogg, Rugg 

Lyman, Gray , 

Buswell and Judd, Monroe 
and Engelhart 

Curtis 

Buchanan and MacPhee, Cole, 
Coleman and Jacques 

Ryan 

Blackstone 

Watson 

Smith and Krueger 

Champneys 

Bibliographie Pedagogique In- 
ternationale 


Miscellaneous guides to educational literature, In the chap- 
ter bibliography certain additional guides of assistance in 
locating information in a variety of educational fields are 


listed: 


Periodical literature 
Educational research in peneral 9 
Personnel literature 

Psychology . 

Teacher training 

Secondary education 
Curriculum . 

Negro education 

Elementary education . 
Occupational research . 
Physical education . 

School administration and research 
Commercial education 
Graduate theses in education . 


Carlson, Witmer and Miller 
Witmer, Whitney and Rugg 
Cowley 

Louttit 

Lancaster 

Manske and Alexander 
Dech 

Cook 

McSwain and Alexander 
Davis 

Sefton 

Alexander 

Gropp 

Derring 
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History - . CIRATA pe ee CAD utcher: 

American history, oils Fala EE aE 

School law... . . . . ~ Edwards 

Speech education . . fei Tauber: 

Office of Education publications . . Witmer and Miller 
Comparative education . . . . Abel 


A good discussion of how to make such guides and a rather 
complete list of educational guides are available.** 

Four other library aids may be mentioned, although strictly 
speaking, they are not guides to current literature. However, 
they can be made to serve somewhat the same purpose, in that 
the topics, terms, and to some extent the problems in the 
fields of education and psychology are indicated. English has 
provided a dictionary of more than 2000 psychological terms, 
while Warren’s later effort includes 10,000 terms.** Haggerty 
has listed 2,584 topics in psychology.** Voegelein has prepared 
a comprehensive list of educational subject headings,°* 

At the risk of repetition, it may be stated that these bibli- 
ographies, abstracts, summaries, and guides have been made 
necessary by the rapid increase in educational literature dur- 
ing the past decade. Critical summaries require research ability 
of high order and are among the most valuable contributions 
to educational literature. Whether on the university campus or 
in the field, the student of education may save himself much 
searching in the library and some mental anguish by becom- 
ing familiar with available guides, bibliographies, and sum- 
maries. To-day there is no reasonable excuse for ignorance 
concerning worth-while work (whether books, monographs, 
bulletins, year-books, periodical articles, or graduate theses) 
previously done in the field of interest, even though such lapses 
may have been viewed with more tolerance in the past than 

85 Carter Alexander, How to Locate Educational Information and Data, op. cit., pp. 
Sera E English, A Student's Dictionary of Psychological Terms, Yellow Springs, 
Ohio: Antioch Press, 1929. Pp. 82. H. C. Warren, Dictionary of Psychology. Boston: 
Houghton Mifflin Co., 1934. 

37 M. E. Haggerty, Topics in Psychology. Minneapolis, Minn.: University of Minne- 
sota Press, 1929, 


38L. Belle Voegelein, List of Educational Subject Headings. Columbus. Ohio: Ohio 
State University Press, 1928. Pp. xiv + 338. 


126 EDUCATIONAL RESEARCH 


in the present or future. Ten years ago most of the biblio- 
graphical and library aids cited above did not exist. 

Evidence of the extent to which research workers employ 
reference materials is found in Louttit’s statement that in 144 
periodicals in psychology and related fields over the period 
1900-1927 there was a total of 2,134 bibliographies.*° 

The preceding section has been concerned with the various 
educational guides useful in making a reasonably rapid, yet 
fairly complete bibliographical survey of a problem. The re- 
mainder of this chapter is devoted to a more highly detailed 
and specialized treatment of the various types of educational 
literature and the sponsoring organizations or agencies. 

Periodical literature in education. Investigators have been 
active in determining the journals most frequently read, the 
type of authorship represented, and the topics treated. Such 
information is of value in directing the reading and study of 
educational workers. 


TABLE V * 


Tue HELPFULNESS or CERTAIN PROFESSIONAL JOURNALS 


Index of Rank in 


Journals helpfulness | helpfulness 


BEKOOR REVIEW n aaae ea enan esoe in OSA naa 45.5 
Educational Administration and Supervision.... 321 2 
American School Board Journal...... ` 21.2 3 


Teachers College Record....... 10.8 4.5 
Journal of Educational Research. 19.8 4.5 
Journal of Education. 18.8 6 
Educational Review . 14.5 7 
School and Society .. 13.3 8 
State Journals of Education Bers 12.2 9 
Education) Sis 20). RENAE ee aO F 10.3 10 


* Rearranged trom author’s data. 


Extent to which journals are read, and guidance for such 
reading. A questionnaire study of significance to secondary- 


39C. M. Louttit, “The Use of Bibliographies in Psychology,” Psychological Review, 
XXXVI (July, 1929), 341-7. 
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school principals sought to discover which of ten designated 
educational journals were read most by senior high-school 
principals. The conclusions of the investigation are based on 
returns from 381 principals located in 375 cities, including all 
the states and the District of Columbia. The periodicals were 
rated in terms of helpfulness in the work of the principals. A 
percentage index of helpfulness was secured by dividing the 
number of times a journal was mentioned as most helpful by 
the total number of times it was being read. The second 
column of Table V gives this index of helpfulness, and the 
last column indicates the rank in helpfulness. 

Kelly and Whitney made a companion study to that reported 
by LaPoe,*t and summarized above, in order to determine the 
educational periodicals read by elementary principals and class- 
room teachers.*? This questionnaire investigation involves 250 
elementary principals and 250 elementary teachers in 156 
cities in forty states and the District of Columbia. In Table VI 
is found a list of professional journals ranked according to 
their helpfulness to three different groups of school workers. 
The figures in the second column of Table VI are taken from 
LaPoe’s study. The data in the third and fourth columns are 
a part of Kelly and Whitney’s findings. A blank space in any 
column means that the periodical involved did not appear in 
the list of the ten journals read most frequently by the group 
in question. 

Eells classifies, according to the journals represented, 2,500 
articles read by 115 students at Stanford University in the 
twenty-eight periodicals which are represented by twenty or 
more articles.t? Approximately forty other journals are reported 
with less than twenty articles each. The five journals read with 
greatest frequency and listed according to rank were School 


40 James L. LaPoe, “The Senior High School Principals’ Professional Magazines,” 
Educational Research Bulletin, VI (September 14, 1927), 259-61. 

41 Ibid. 3 

42E. L, Kelly and F. L. Whitney, “Educational Magazines Read by Five Hundred 
Elementary-School Principals and Classropm Teachers,” Elementary School Journal, 
XXIX (November, 1928), 176-80. 

43 W. C., Eells, “What Professional Magazines Do Prospective Educators Read?” 
School and Societ» XXIX (April 6, 1929), 446-8- 
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and Society, School Review, Sierra Educational News (state 
journal), Educational Review (now combined with School and 
Society), and Journal of the National Education Association. 


TABLE VI 


Epucationar Macazines RANKED as to HELPFULNESS to Hicn-Scuoor 
PRINCIPALS, ELEMENTARY-SCHOOL PRINCIPALS, AND ELEMENTARY- 
ScHoor TEACHERS 


High- |Elementary-|Elementary- 
Magazine School School School 
Principals | Principals | Teachers 


American Educational Digest.. 
American School Board Journal. 
Child-Welfare Magazine .. 
Education 
Educational Administration ree Super. 

vision 
Educational Review... 
Elementary Schooi Journal. 
Industrial-Arts Magazine . 
Journal of Education....... 
Journal of Educational Method 
Journal of Educational Research. 
Journal of the National Education 2e 

SOGIGLEOMI A E OA A TA 
Normal Instructor and Primary Plans..| .. 
Primary Education—Popular Educator.| .... 10 


3 


School Review ... I rics 
School and Society.. 8 eerie 8 

State Journals of Education. k, 9 9 I0 

Teachers College Record.............. 4.5 8 


Eells also asked the preferences of college juniors and seniors 
in the course, “Introduction to Education,” at Stanford, con- 
cerning three journals to be read regularly.*t More than one- 
half of the class chose School and Society, while the Journal of 
the National Education Association and the School Review 
received the votes of one-third of the students. The journals 
selected by approximately one-sixth of the class were School 
Executives Magazine, Progressive Education, Nation’s Schools, 


44 W. C. Eells, “Are Educational Journals Worth Reading,” Journal of the National 
Education Association, XIX (May. 1930). 1s¢-40, 
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California Quarterly of Secondary Education (now California 
Journal of Secondary Education), American School Board 
Journal, Journal of Adult Education, and Sierra Educational 
News. 

Reports from 253 members of the National Society for the 
Study of Education and the National Society of College Teach- 
ers of Education, who were asked in 1917 to name from one 
to five educational journals desirable for a public’ library, 
recommended in the order named: School and Society, School 
Review, Educational Review, Elementary School Journal, Edu- 
cational Administration and Supervision, American School 
Board Journal, Journal of Education, Journal of Educational 
Psychology, Education, and Teachers College Record“ Of 
course, since that date the Educational Review has combined 
with School and Society, and other important journals have 
been established. 

There is space only to list in the chapter bibliography certain 
other informative analyses of periodical literature: topics 
treated or type of authorship represented (Ashbaugh, Barr, 
Becker, Collins, Cornelinson, Davis and Ballard, Franke and 
Davis, Goodenough, Louttit, Martin, Shannon, Strang); sug- 
gested lists of journals or guidance for such reading (Alexan- 
der, Clark, Copeland, Dillman, Lyle, St. John) ; and the devel- 
opment of educational journalism during the past century 
(Davis). 

Of course, the thoughtful reader will not allow the foregoing 
analyses of the extent to which certain journals are read to 
distract his attention unnecessarily from many other useful 
periodicals named subsequently in this chapter. As a matter 
of fact, certain of the journals most valuable to research work- 
ers in psychology and education have at this writing a sub- 
scription list numbered in hundreds rather than thousands 
or tens of thousands. At the same time it is quite common 
for important reports of investigations to be held in editorial 
files for a year or more before appearing in print. This con- 

45 School SUE VI (October 13, 3917), 433-4. 
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dition might be remedied in part by increasing the number 
of subscribers and, as a result, enlarging the space available for 
research reports. In particular those interested in psycho- 
logical journals are attempting to work out some plan for re- 
ducing the paralyzing lag in publication, which has taken 
tangible form in the Journal of Psychology. This periodical 
and the Journal of Experimental Education are published on 
a codperative basis, with the author sharing in the costs in- 
volved. 

General educational journals and periodical literature in 
subject-matter fields. In a preceding section of this chapter 
a detailed account of guides to educational literature has been 
given. At this time it is in order to characterize more fully 
the field of periodical literature. 

Educational journals of general interest. The handbooks or 
year-books of the Educational Press Association of America, 
Washington, D. C., which include classified lists of educa- 
tional periodicals, are the most complete available sources of 
information on this subject. The Education Index, as well as 
other periodical indexes, publishes lists of the journals repre- 
sented. At one time a list (discontinued after 1932) of educa- 
tional periodicals received by the library of the Office of Edu- 
cation was published in its annual educational directory. 

The extent to which different types of educational organiza- 
tions and institutions publish journals representing their special 
fields of interest is represented in part by examining the classi- 
fication of journals employed in the year-books of the Educa- 
tional Press Association of America: general in scope and 
national in circulation, issued as organs of state education 
associations, issued by state departments of education, state 
or regional journals issued by individuals or corporations, 
local journals issued under the auspices of city boards of edu- 
cation, local journals issued as organs of professional organiza- 
tions, and journals of institutions for the education of teachers. 

In the immediately following list of journals, only those of 
rather general interest, as distinguished from subject-matter 
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fields, and those of national scope, will be given. In a separate 
compilation will be listed those periodicals devoted to subject- 
matter fields. Some of the journals of general scope given 
. below are organs of certain educational associations, and are so 
indicated. Those which print a considerable number of factual 
studies and experiments in contrast to theoretical, speculative, 
or descriptive discussions will be preceded by an asterisk. 


American Journal of Psychology. Cornell University, Ithaca, New 
York. Quarterly. 

| * American School Board Journal. Bruce Publishing Co., 407 East 

| Michigan Street, Milwaukee, Wisconsin. Monthly. 

Bulletin of the American Association of University Professors, 

| American Association of University Professors, 744 Jackson 
Place, N.W., Washington, D.C. 8 numbers. 

Bulletin of the Association of American Colleges. Association of 
American Colleges, 111 Fifth Avenue, New York City. 4 
numbers. 

* California Journal of Secondary Education. California Society for the 
Study of Secondary Education, Berkeley, California. 8 numbers. 

Catholic Educational Review. Catholic Education Press, 1326 Quincy 
Street, N.E., Washington, D.C. 10 numbers. 

Chicago Schools Journal. Board of Education, Chicago Normal Col- 
lege, Sixty-eighth Street and Stewart Avenue, Chicago, Illinois. 
ro numbers. . 

Child Development. Williams and Wilkins Co., Mt. Royal and Guil- 
ford Avenues, Baltimore, Maryland. Quarterly. 

Childhood Education. Association for Childhood Education, 1201 
Sixteenth Street, N.W., Washington, D.C. 9 numbers. 

Education. The Palmer Co., 120 Boylston Street, Boston, Massa- 
chusetts, ro numbers. 

Education Digest. Box too, Ann Arbor, Michigan. ro numbers. 

Educational Abstracts. 230 Fifth Avenue, New York City. 5 numbers. 

* Educational Administration and Supervision. Warwick and York, 10 
E. Centre Street, Baltimore, Maryland. 9 numbers. 

* Educational Law and Administration. 9 West Fourth Street, Cincin- 
nati, Ohio. Quarterly. 

* Educational Method. Department of Supervisors and Directors of 
Instruction, 525 West One Hundred and Twentieth Street, New 
York City. ọ numbers. (Formerly Journal of Educational 
Method.) 

Educational Record. American Council on Education, 744 Jackson 
Place, N.W., Washington, D.C. Quarterly. 
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* Educational Research Bulletin. Ohio State University, Columbus 
Ohio, 9 numbers. 
* Elementary School Journal. Department of Education, University of 
Chicago, Chicago, Illinois. 1o numbers. 
Journal of Abnormal and Social Psychology. Psychological Review 
Co., Princeton, N. J. Quarterly. i 
Journal of Adult Education. American Association for Adult Educa- 
tion, 6o East Forty-second Street, New York City. Quarterly. 
Journal of the American Association of University Women. 1634 Eye 
Street, N.W., Washington, D. C. Quarterly. 
* Journal of Applied Psychology. Williams and Wilkins Co., Mount 
Royal and Guilford Avenues, Baltimore, Maryland. Bimonthly. 
Journal of Education. New England Publishing Co., 6 Park Street, 
Boston, Massachusetts, 20 numbers. 
* Journal of Educational Psychology. Warwick and York, 10 E. Centre 
Street, Baltimore, Maryland, 9 numbers. 
* Journal `of Educational Research, Public School Publishing Co., 
Bloomington, Illinois. 9 ‘numbers. 
* Journal of Educational Sociology. American Viewpoint Society, 13 
Astor Place, New York City. 9 numbers. 
* Journal of Experimental Education, Journal Publishing Co., Madison, 
Wisconsin. Quarterly. 
*Journal of Experimental Psychology. Psychological Review Co., 
Princeton, New Jersey. Bimonthly. 
Journal of General Psychology. Clark University Press, Worcester, 
Massachusetts. Quarterly. 
* Journal of Higher Education. Ohio State University, Columbus, Ohio. 
9 numbers. 
* Journal of Negro Education. Howard University, Washington, D.C. 
Quarterly. 
* Journal of Psychology, Worcester, Massachusetts. 
Journal of Social Hygiene. American Social Hygiene Association, 50 
West Fiftieth Street, New York City. 9 numbers. 
Journal of Social Psychology. Clark University Press, Worcester, 
Massachusetts. Quarterly. 
Journal of the National Education Association, 1201 Sixteenth 
Street, N.W., Washington, D.C. 9 numbers, 
* Junior College Journal. American Association of Junior Colleges and 
School of Education, Stanford University, California. 8 numbers. 
Junior-Senior High School Clearing House. School of Education, New 
York University, New York City. (Formerly Junior High School 
Clearing House.) 9 numbers. 
Mental Hygiene. 50 West Fiftieth Street, New York City. Quarterly, 
* Nation’s Schools. 919 North Michigan Avenue, Chicago, Illinois. 
Monthly. 
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North Central Association Quarterly. University High School Build- 
ing, Ann Arbor, Michigan. Quarterly. 

* Peabody Journal of Education. George Peabody College for Teachers, 
Nashville, Tennessee. Bimonthly. 

* Pedagogical Seminary and Journal of Genetic Psychology. Clark Uni- 
versity, Worcester, Massachusetts. Quarterly. 

* Personnel Journal. Personnel Research Federation, Williams and Wil- 
kins Co., Mount Royal and Guilford Avenues, Baltimore, Mary- 
land. Bimonthly, (Formerly Journal of Personnel Research.) 

* Phi Delta Kappan. 1180 East Sixty-Third Street, Chicago, Illinois. 
9 numbers. Published by a professional educational fraternity, 
but is of general interest 

Progressive Education. Progressive Education Association, 310 West 
Ninetieth Street, New York City. 8 numbers. 

* Psychological Abstracts. American Psychological Association, Prince 
and Lemon Streets, Lancaster, Pennsylvania. 13 numbers. 

* Psychological Bulletin. American Psychological Association, Prince 
and Lemon Streets, Lancaster, Pennsylvania. 10 numbers. 
Psychological Review. Psychological Review Co., Princeton, N. J. 

6 numbers. 

* Research Bulletin. National Education Association, Research Divi- 
sion, 1201 Sixteenth Street, N.W., Washington, D. C. 5 numbers. 

* Review of Educational Research. American Educational Research 
Association, 1201 Sixteenth Street, N.W., Washington, D. C. 5 
numbers, 

* School and Society. Science Press, Grand Central Terminal, New 
York City. Weekly. 

School Executive. 1126-28 Q Street, Lincoln, Nebraska, Monthly. 
(Formerly American Educational Digest.) 

School Life, United States Office of Education, Washington, D. G 
ro numbers. 

* School Review. Department of Education, University of Chicago, 
Chicago, Illinois. ro numbers. 

Social Frontier. 66 West Eighty-eighth Street, New York City. 0 
numbers. 

Social Science Abstracts. Social Science Research Council, Columbia 
University, New York City. Monthly. (Suspended publication in 
1932. 

Bes Sms Social Research. University of Southern California, 
Los Angeles, California, Bimonthly. 

*Teachers College Record. Teachers College, Columbia University, 
New York City. 8 numbers. 

Teachers Journal and Abstract. Colorado State Teachers College, 
Greeley, Colorado. 9 numbers. (Suspended publication in 1932, 
four years before Educational Abstracts began.) 
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Educational periodicals and organizations in subject-matter 
fields. It seems desirable to enumerate for the benefit of 
teachers and supervisors of the various school subjects the 
professional journals, year-books, proceedings, reports, and 
organizations in the twelve major divisions of instruction. 
Therefore, each of the twelve divisions (found in the appen- 
dix) includes a selected list of periodicals, foundations and 
boards, and professional associations in the field represented. 
Wherever possible, a permanent address is given. Information 
concerning the addresses of organizations which do not have 
a permanent central office can be secured from the annual 
educational directory of the United States Office of Education, 
Additional information concerning most of the publications 
can be found in the year-books of the Educational Press As- 
sociation of America. 

Books, monographs, year-books, bulletins, surveys, and 
reports. Although guides to the various types of educational 
literature have been cited earlier in this chapter, it seems 
desirable to describe in greater detail the broad field of pub- 
lications represented by bound volumes in the form of books, 
monographs, year-books, surveys, proceedings, and reports. 

Beginning in November, 1934, the Education Index initiated 
publication of a list of the series (such as textbooks, mono- 
graphs, and bulletins) indexed as separate publications. It also 
includes: a directory of publishers; a check-list of professional 
books; a check-list of publications of institutions, associations, 
and foundations; and a check-list of documents and reports. 

Publications of teacher-training institutions. A large num- 
ber of research studies in education, in some instances Doc- 
tors’ dissertations, appear in the publications of the larger 
schools of education. By far the largest and best known series 
of publications in the following list is that of Teachers College, 
Columbia University, which at this writing includes more 
than six hundred monographs, and consists chiefly of Doctors’ 
studies. A series, relatively limited in number, but of high 
quality, is the Supplementary Educational Monographs. Uni- 
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versity of Chicago. No attempt has been made to repeat the 
names of all educational journals sponsored by teacher-train- 
ing institutions, especially if previously listed in this chapter 
as of general or national scope. 


Catholic University of America, Washington, D. C—Catholic Educa- 
tional Review, and Educational Research Bulletin, continued as 
Educational Research Monographs. 

Clark University, Worcester, Massachusetts—psychological mono- 
graphs and journals. 

Colorade State College of Education, Greeley, Colorado—Bulletins and 
Education Series. 

George Peabody College for Teachers, Nashville, Tennessee—Peabody 
Journal of Education, Contributions to Education, and Field 
Studies of the Division of Surveys and Field Studies. 

Harvard University, School of Education, Cambridge, Massachusetts— 
Bulletins, Monographs, Studies, and Inglis Lectures. 

Indiana University, Bureau of Codperative Research, Bloomington, 
Indiana—Builletins, and Bulletin of the School of Education. 

Johns Hopkins University, Baltimore, Maryland—Studies in Edu- 


cation. 

Kansas State Teachers College, Emporia, Kansas—Studies in Edu- 
cation, 

Northwestern University, Evanston, Illinois—Contributions to Edu- 
cation. 


Ohio State University, Bureau of Educational Research, Columbus, 
Ohio—Bulletins, Monographs, Journal of Higher Education, and 
Educational Research Bulletin. 

Pennsylvania State College, State College, Pennsylvania—Studies in 
Education. 

Purdue University, Lafayette, Indiana—Educational Reference Cir- 
cular, and Studies in Higher Education. 

Rutgers University, New Brunswick, New Jersey—Studies in Educa- 
tion. 

Stanford University, California—School Economy Series. 

State University of Iowa, Iowa City, Ilowa—Bulletins, Monographs, 
and Studies—(a) Iowa Child Welfare Research Station, and (b) 
College of Education Series. 

Stephens College, Columbia, Missouri—Zducational Service Series. 

Teachers College, Columbia University, New York City—Teachers 
College Record, Contributioas to Education, Lincoln School Mono- 
graphs, and Yearbooks of the International Institute. 

University of California, Bureau of Research in Education, Berkeley, 
California—Publications in Education. 
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University of California at Los Angeles—Publications in Education, 
Philosophy, and Psychology. 

University of Chicago, Department of Education, Chicago, Ilinois— 
Supplementary Educational Monographs, Laboratory Schools Pub- 
lications, School Review, and Elementary School Journal. 

University of Illinois, Bureau of Educational Research, Urbana, Illinois 
—Research Bulletins and Research Circulars. 

University of Kansas, Lawrence, Kansas—Kansas Studies in Educa- 
tion. 

University of Kentucky, Lexington, Kentucky—Bulletins. 

University of Michigan, Ann Arbor, Michigan—Bureau of Educational 
Reference and Research Bulletins and School of Education Bul- 
letin. 

University of Minnesota, College of Education, Minneapolis, Minne- 
sota—Monographs, Studies, and Bulletins. 

University of Nebraska, Lincoln, Nebraska—Educational Research 
Record, Educational Monographs. 

University of New Mexico, Albuquerque, New Mexico—Education 
Series, Bulletin. 

University of North Carolina, Chapel Hill, North Carolina—High 
School Journal, Bulletins. 

University of North Dakota, Grand Forks, North Dakota—Bulletins, 
School of Education Record. 

University of Oregon, Eugene, Oregon—Studies in College Teaching, 
Bulletin. 

University of Pennsylvania, Philadelphia, Pennsylvania—Educational 
Outlook. 

University of Southern California, Los Angeles, California—Educa- 
tional Monographs. 

University of Virginia, University, Virginia—Secondary Education in 
Virginia. 

University of Washington, Seattle, Washington—Publications in Social 
Sciences, School of Education Record. : 

University of Wisconsin, Bureau of Educational Research, Madison, 
Wisconsin—Bulletins, 

Western Reserve University, Cleveland, Ohio—Curriculum Laboratory 
Bulletin. 

Yale University, New Haven, Connecticut—Studies in Religious Edu- 
cation. 


Publications of state and city school systems. State and city 
manuals, courses of study, school-board reports, superin- 
tendents’ reports, etc., are valuable sources of information for 
the student of education and the worker in the public-school 
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field, especially in the areas of administration, testing, cur- 
riculum, and current statistics. Although it is not possible to 
include a list of the city bureaus of educational research, school 
boards, and state departments which publish useful mono- 
graphs, bulletins, and pamphlets, the reader may be referred 
to an article in the Teachers Journal and Abstract for such in- 
formation.** Also the year-books of the Educational Press As- 
sociation of America are of assistance in identifying the agen- 
cies publishing periodicals. Information is available for those 
interested in the organization and activities of city bureaus of 
research.“ The 1932 educational directory of the Office of 
Education includes an extended list of directors of educational 
research in state departments, state education associations, 
city school systems, universities and colleges, teachers colleges 
and normal schools, and bureaus of child development. In 
1930 the Journal of Educational Research established in its 
Department of Research News and Communications a series 
of descriptions of the work and organization of university, 
state, and city bureaus of educational research; at intervals 
progress reports for individual bureaus appear. 

Under the direction of H. B. Bruner at Teachers College, 
Columbia University, a comprehensive collection of courses of 
study has been assembled and rated or evaluated. The Society 
for Curriculum Study has begun the publication of an annual 
selected list of courses Of study, and the Education Index 
includes a list of courses of study in the monthly issues. From 


46 Frederick L. Whitney and Earle U, Rugg, “Sources of Current Literature in Edu- 
cation,” Teachers Journal and Abstract, I (January, 1926), 47-53. 

47H. B. Chapman, Organized Research in Education with Special Reference to the 
Bureau of Educational Research. Bureau of Educational Research Monographs, No. 7. 
Columbus, Ohio: Ohio State University Press, 1927. Pp. x +222. 

Edith A. Wright, Organization and Functions of Research Bureaus in City School 
Systems. Office of Education Leaflet, No. 2, 1931. Washington: Office of Education, 1931. 

W. H. Ziegel, Research in Secondary ‘Schools, Office of Education Bulletin No. 17, 
1932. National Survey of Secondary Education Monograph, No. 15. Washington: Office 
of Education, 1933. Pp. vi + 72. $ 

48H. B. Bruner, “Selected List of Courses of Study Published 1932-1933," Society 
for Curriculum Study News Bulletin, IV (November 27, 1933), 2-7- 
, “Outstanding Courses of Study of 1934,” Society for Curriculum Study News 
Bulletin, VI (January 15, 1935), 2-8- 
and Mabel V. Cassell, “Annual List of Outstanding Courses of Study,” Cur- 
riculum Journal, VI (December 2, 1935), 1-9- 
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time to time the United States Office of Education publishes 
a list of courses of study on file in its library.*® 

Survey reports. National foundations and societies, denomi- 
national groups, regional organizations, national and state gov- 
ernments, teachers associations in subject-matter fields, and 
individual school systems have promoted intensive surveys of 
school facilities and conditions, which provide valuable data 
concerning public-school systems and higher institutions. In 
fact, more than one-third of the states, and many cities and 
counties, have conducted surveys of their school systems. 
Caswell tabulates the growth of this movement since 1910 
by years and indicates the organizations which are active in 
making such studies.°° A standard work on this subject in- 
cludes a selected list of surveys of especial interest to public- 
school workers." A bulletin of the United States Office of 
Education includes a summary of surveys of higher education, 
as well as of city and rural surveys.** The extent of this move- 
ment in higher institutions is indicated in part by another re- 
view of nineteen curriculum surveys." A helpful topical analy- 
sis of the content of 234 school surveys, and an extensive list 
of surveys and references on surveys, are available.°* Eaton 
and Harrison have provided a bibliography of 2775 social 
surveys.“ The technique of the school survey will be discussed 
in later chapters (VII and VIII). 

49 Edith A. Wright, List of Courses of Study for Elementary and Secondary Schools, 
1930-1935. United States Office of Education Circular, No. 139. Washington: Office of 
Education, March, 1935. Pp. 46. 

50H. L. Caswell, City School Surveys. Teachers College Contributions to Education, 
No, 358. New York: Teachers College, Columbia University, 1929. 

» “Is the School Survey Movement Dead?” Peabody Journal of Education, VII 
(September, 1929), 108-r4. 

51 Jesse B. Sears, The School Survey, pp. 429-33. Boston: Houghton Mifflin Co., 1925. 

52 Arthur J. Klein, Walter S. Deffenbaugh, Timon Covert, and Edith A, Lathrop, 
Educational Surveys. Bureau of Education Bulletin, No. r1, 1928. Washington: Bureau 
of Education, Pp. 68. 

53 L. E. Blauch, “Curriculum Surveys in Higher Education,” Journal of Higher Educa- 
tion, IV (May, 1933), 255-60. 

54H. L. Smith and E. A. O'Dell, Bibliography of School Surveys and of References 
on School Surveys. Bulletin of the School of Education, Indiana University, Vol. VIII, 
Nos. r and 2. Bloomington, Ind.: Indiana University, 1931. Pp. 212. 

Topical Analysis of 234 School Surveys, Bulletin of the School of Education, Indiana 
University, Vol. III, No. 4. Bloomington, Ind.: Indiana University, 1927. Pp. 112. 


55 Allen Eaton and S. M. Harrison, A Bibliography of Social Surveys. New York: 
Russell Sage Foundation, 1931. Pp. 468. 
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Selected major investigations. Since the survey movement 
and other large-scale studies of national significance have re- 
ceived such widespread attention during the past decade, sev- 
eral of these investigations will be listed in the chapter bib- 
liography; they indicate a valuable addition to educational 
literature, as well as a type of planning and codérdination in 
research, These selected studies may be located in the bibli- 
ography under the name of the author or agency represented 
and include the fields, investigators, and sponsoring agencies 


named in the following list: 


Land-grant colleges 
Secondary education 
Teacher education 


School finance 


Social studies 


English usage ó 
Child health and protection 


Social trends 


Character education 
Nature and nurture 


Reorganization of the American edu- 
cational system 


Modern foreign ae 
Latin : : 


Mathematics 

Adult education 

Reading 

Genius y 

Measurement Se intelligence 
Class size 

Science teaching 


Klein 

Koos 

Evenden, Charters and Waples, 
Betts 

Educational Finance Inquiry, 
Alexander and Covert 

Commission on the Social 
Studies of the American 
Historical Association 

O’Rourke 

Whitehouse Conference on 
Child Health and Protection 

President’s Committee on So- 
cial Trends 

Hartshorne and May 

National Society for the Study 
of Education 

Committee on Administrative 
Units of the Commonwealth 
Fund 

Henmon, Fife 

Lucasse, Advisory Committee 
of the American Classical 
League 

Young 

Carnegie Corporation 

University of Chicago 

Terman 

Thorndike 

Hudelson 

Freeman 
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United States Office of Education. The numerous publica- 
tions, surveys, and statistical reports of this agency cover a 
wide range of problems and include the best available single 
source of information on current educational conditions, con- 
temporary educational literature, educational statistics, and im- 
portant movements in education. At intervals a publication of 
the Office of Education indexes by year, author, title, and sub- 
ject the various bulletins issued over a given period of years, 
for example, 1910 to 1927.°° The annual reports of the Commis- 
sioner of Education list the projects completed by the Office 
of Education and those under way. Helpful accounts of the 
publications and the excellent library of the Office of Educa- 
tion are also available.’ Under Commissioner Cooper’s admin- 
istration a reorganization of the responsibilities of the Office 
of Education took place, based upon a conception of the 
Office of Education as a research organization rather than an 
administrative agency. Therefore, its policy is to abandon and 
transfer to other agencies, in so far as possible, such functions 
as the supervision of the education of natives and the control 
of the reindeer industry in Alaska. The Office aims to concen- 
trate its attention upon fact-finding, surveys, and research in 
the many fields of education. In keeping with this new em- 
phasis the name of the former Bureau of Education has been 
changed to Office of Education. Evidence of this new emphasis 
is found in the four comparatively recent national surveys of 
Land-Grant Colleges, Secondary Education, Teacher Education, 
and School Finance, sponsored by the Office. The Federal 
Board for Vocational Education is now a division of the Office 
of Education. 7 

56 Edith A. Wright and Mary S. Phillips, Bulletins of the Bureau of Education, 
1910-1927. Bureau of Education Bulletin, No. 17, 1928, Washington: Bureau of Educa- 
Mion erena “Ri McCabe and H.R, Evans, “The Library of the Office of Education 
and Some of Its Special Collections,” School Life, XV (November, 1929), 47-9. 

Sabra W. Vought, “The Library of the Federal Office of Education,” Peabody Journal 
of Education, XU (July, 1934), 21-5. 

Eleanor M. Witmer and Margaret C. Miller, “United States Office of Education Serial 
Publications,” Teachers College Record, XXXIV (January, 1933), 302-11. 


58 W. J. Cooper, “Some Responsibilities of the United States Bureau of Education.” 
Educational Record (July, 1929), 184-99. 
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Fortunately, a number of valuable guides for using the pub- 
lications of the Office and accounts of its organization and 
activities are available.*° 

Government documents. Other government documents of a 
less specific educational character than those published by the 
Office of Education have been issued in large numbers. They 
cover a wide range of political, economic, and social problems 
touching federal, state, and local (city, village, county, and 
township) areas. Detailed assistance is available for canvassing 
the various types of journals and proceedings, directories and 
registers, rules and regulations, laws, reports, and special stud- 
ies. In the bibliography of this chapter are listed certain 
guides for canvassing government documents. Chapter VI of 
the present volume includes an extended analysis of pro- 
cedure in legal research as related to educational problems. 

Year-books and other publications of departments of the 
National Education Association. Attention will be directed 
later in this chapter to professional and Jearned organizations 
which issue important year-books, bulletins, or journals, but 
it seems advisable at this time to indicate briefly the topics 
treated in certain series of year-books and publications of 
especial interest to field workers in education. The annual 
volume of Proceedings of the National Education Association 
contains abstracts of the addresses presented at the meeting 
of this organization. In addition to a series of stimulating ar- 
ticles, the Journal of the National Education Association 
usually includes significant current statistics, for example, 


59 Carter Alexander, How to Locate Educational Information and Data, op. cit, pp- 


123-39- by A 
Bulletins of the Bureau of Education from 1906-1927. Bureau of Education Bulletin 


No. 17, 1928. Washington: Bureau of Education, 1928. 
J. W. Cooper, “The Office of Education,” Scientific Monthly, XXXVI (February, 
1933), 121-30. 
n of Publications of the U. S. Bureau of Education, 1867-1910. Bureau of Educa- 
tion Bulletin, No. 3, 1910. Washington: Bureau of Education, 1910. fi 
D. H. Smith, The Bureau of ‘Education—Its History, ‘Activities, and Organization. 
Service Monographs of the United States Government, No. 14. Baltimore, Maryland: 
Johns Hopkins Press, 1923. Pp. xiv + 158. 


60 Carter Alexander, op. Cit., pp. 113-22. 3 
Index to the Reports of the ‘Commissioner of Education, 1867-1907. Bureau of Educa- 


tig Rtctin, No. 7. 1909. Washington: Bureau of Education, 1909. Pp. 103- 
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school statistics concerning the various countries of the world, 
educational developments in the various states, membership 
Statistics of the Departments of Superintendence, Secondary 
School Principals, and Elementary School Principals, figures 
on the growth of the National Education Association and state 
education associations, information on the growth of allied 
education associations, and figures on the growth of high schools 
by states during the present century. 

Valuable contributions in the curriculum field have been 
made by the year-books of the Department of Superintendence. 
This association, beginning the series in 1923, has published 
year-books under the following titles: The Status of the Su- 
perintendent (192 3), The Elementary School Curriculum 
(1924), Research in Constructing the Elementary School Cur- 


The Department of Secondary School Principals began 
publication of its year-books in 1917 and since January, 1926, 
has issued four bulletins each year. 

The Department of Elementary School Principals publishes 
its Bulletin five times a year, one number of which is a year- 
book. Its fourteenth year-book was issued in 1935. 

The Department of Classroom Teachers in 1936 published 


The American Educational Research Association, a depart- 


n 
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ment of the National Education Association, publishes the 
Review of Educational Research, which has been described 
earlier in this chapter. 

The Research Division of the National Education Associa- 
tion in 1923 began a series of bulletins, the Research Bulle- 
tin, published five times a year, which las proved of unusual 
value to public-school workers because of the statistics and 
summaries of professional literature made accessible in this 
way. It also issues a series of circulars on a variety of educa- 
tional problems. 

The foregoing and other departments are represented in 
the annual volume of proceedings of the Association. Some of 
these departments, not specifically cited above, issue bulletins 
and even year-books, although space does not permit a listing 
of such publications here. Detailed information concerning 
the proceedings, bulletins, and year-books of the various de- 
partments of the Association and descriptions of its organiza- 
tion and activities are available elsewhere.®* 

National Society for the Study of Education. Likewise it 
seems worth while to mention here the specific work of the 
National Society for the Study of Education, whose year- 
books are probably the most pretentious and most carefully 
worked out of any issued by existing educational organiza- 
tions. The topics which have been discussed or investigated 
by this society since 1920 are concerned with the problems of 
new materials of instruction, the education of gifted children, 
silent reading, intelligence testing, English composition, social 
studies, vocational education, adapting the schools to individ- 
ual differences, safety education, extracurricular activities, 
curriculum-making, possibilities and limitations of training, 
preschool and parental education, arithmetic, rural education, 
school textbooks, science teaching, changes in liberal arts col- 
leges, geography, school buildings, the activity movement, edu- 

61 Erwin S. Selle, The Organization and Activities of the National Education Associa- 
tion, Teachers College Contributions to Education, No. 513. New York: Bureau of 


Publications, Teachers College, Columbia University, 1932. 
Carter Alexander, op. cit., pp. 140-51. 
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cational diagnosis, grouping of pupils, and music education, 
with committees at work on international understanding and 
the curriculum, reading, organization of higher education, and 
education as a science. In the current year-book of a given 
professional organization a list of preceding year-book topics 
and of those under way will usually be found. 

American educational and learned associations other than in 
subject-matter fields. A selected list of educational and learned 
organizations interested to a greater or less extent in the gen- 
eral problems of education, as compared with subject-matter 
associations, is given below, Those organizations preceded by 
an asterisk issue important publications such as a year-book, 
journal, volume of Proceedings, or bulletin, As has been sug- 
gested, the current year-book of a given group usually lists the 
titles of Preceding publications, as well as of reports in prog- 
ress. Earlier in this chapter may be found more detailed com- 
ment on the contributions of certain major year-books. Another 
part of this volume presents a classification of educational 


tivities. 
Detailed information is available in published sources con- 
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cerning the organization of research agencies and institutions 
in the humanistic and social sciences.” 


American Association for Adult Education, 60 East Forty-Second 
Street, New York City, Monographs, Journal. Quarterly. 

* American Association for the Advancement of Science, Washington, 
D. C., Proceedings, Science (weekly), Scientific Monthly. 

* American Association of Collegiate Registrars, Proceedings, Bulletin, 
Quarterly. 

* American Association of Junior Colleges, Proceedings, Junior College 
Journal. 9 numbers. 

American Association of Technical High Schools and Institutions. 

* American Association of University Professors, Bulletin. 8 numbers. 
Also occasional special reports. 

* American Association of University Women, Journal (quarterly), 
Monographs. 

* American Council of Learned Societies Devoted to Humanistic 
Studies, 907 Fifteenth Street, N.W., Washington, D. C., Bulletins. 

* American Council on Education, 744 Jackson Place, Washington, 
D. C., Educational Record. Quarterly. 

* American Educational Research Association, Review of Educational 
Research. 5 numbers. 

* American Federation of Organizations for the Hard of Hearing, 
Proceedings. 

* American Federation of Teachers, Bulletin. Monthly. 

* American Psychological Association, Psychological Review, Psycho- 
logical Bulletin, Journal of Experimental Psychology, Psycho- 
logical Monographs, Journal of Abnormal and Social Psychology, 
Psychological Abstracts, Psychologiċal Index. 

* Association of American Colleges, 111 Fifth Avenue, New York City, 
Bulletin. Quarterly. 

* Association of American Colleges and High Schools of the Society of 
Jesus, Annual Reports. 

* Association of American Universities, Proceedings. 

* Association of Colleges and Secondary Schools of the Southern 
States, Proceedings, Reports. 

* Association of Colleges for Negro Youth, Minutes. 

Association of Departments of Education in State Universities and 
Land-Grant Colleges. 


62W. P. Gee, Social Science Research Organisation in American Universities anc 
Colleges. New York: D. Appleton-Century Co., 1934. Pp. 276. 

F. A. Ogg, Research in the Humanistic and Social Sciences. New York: The Century 
Co., 1928. Pp. 454. 

W. E. Spahr and R. J. Swenson, Methods and Status of Scientific Research, Chapten, 
XV-XVIII. New York: Harper and Bros., 1930, 

* Indicates a publication representing the interests of the given group. 
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* Association of Governing Boards of State Universities and Allied 
Institutions, Proceedings. 
* Association of Land-Grant Colleges and Universities, Proceedings. 
* Association of Private School Teachers, Annual Reports. 
* Association of University and College Business Officers of the Eastern 
States, Minutes. 
* Association of Urban Universities, Proceedings, Reports. 
* College Entrance Examination Board, 431 West One Hundred and 
Seventeenth Street, New York City, Annual Reports, Bulletins. 
Head Masters’ Association. 
Head Mistresses’ Association of the East. 
Head Mistresses’ Association of the Middle West. 
Head Mistresses’ Association of the Pacific Coast. 
* Institute for Education by Radio, Yearbooks. 
* Middle States Association of Colleges and Secondary Schools, Pro- 
ceedings. 
National Association of College Deans and Registrars in Negro 
Schools, 
* National Association of College Women, Journal. 
* National Association of Deans and Advisors of Men, Proceedings. 
National Association of High School Supervisors and Inspectors. 
* National Association of Principals of Schools for Girls, Yearbooks. 
* National Association of Public School Business Officials, Bulletins. 
* National Association of State Universities in the United States of 
America, Proceedings. 
* National Catholic Educational Association, Bulletin, Quarterly. 
* National Committee on Research in Secondary Education, Bulletins, 
Reports. 
* National Congress of Parents and Teachers, Yearbooks, Proceedings. 
* National Education Association, 1201 Sixteenth Street, N.W., Wash- 
ington, D. C., Journal (9 numbers), Annual Proceedings, and 
Research Bulletin. 5 numbers, 
* National Education Association, Departments of: 
Administrative Women. 
* Adult Education, Adult Education, 
Art Education. 
* Business Education, Quarterly, 
* Classroom Teachers, Yearbooks, Bulletins. 
* Deans of Women, Bulletins, Yearbooks. 
* Educational Research, Review of Educational Research. 
* Elementary-School Principals, Yearbooks, Bulletins. 
Kindergarten Primary Education, 
* Lip Reading, Bulletins. 
Music Education. 
* Rural Education. Bulletins (Yearbooks), 
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School Health and Physical Education. 
Science Instruction. 
* Secondary Education, Bulletins. 
* Secondary School Principals, Yearbooks, Bulletins. 
* Social Studies, Social Studies, Yearbooks. 
Special Education. 
* Superintendence, Yearbooks, Bulletins, Official Reports. 
* Supervisors and Directors of Instruction, Yearbooks, Educa- 
tional Method. 
* Supervisors and Teachers of Home Economics, Bulletins. 
* Teachers Colleges, Yearbooks. 
* Visual Instruction, Educational Screen and Visual Instruc- 
tion. 
National Council of Education (strictly speaking, not a de- 
partment). 
National Honor Society of Secondary Schools. 
* National Research Council, Division of Education Relations, B and 
Twenty-first Streets, N.W., Washington, D. C., Bulletins. 
* National Society for Curriculum Study, Curriculum Journal. 
* National Society for the Study of Education, Yearbooks. 
* National Society for the Study of Educational Sociology, Yearbooks. 
* National Society of College Teachers of Education, Yearbooks. 
* National Vocational Guidance Association, Vocational Guidance 
Magazine. 8 numbers. 
New England Association of College Teachers of Education. 
* New England Association of College and Secondary Schools, Pro- 
ceedings, Reports. 
* New England College Entrance Certificate Board, Annual Reports. 
New England Vocational Guidance Association. 
* North Central Association of Colleges and Secondary Schools, North 
Central Association Quarterly. 
* Northwest Association of Secondary and Higher Schools, Annual 
Reports, Bulletins. 
Private School Association of the Central States. 
* Progressive Education Association, 310 West Ninetieth Street, New 
York City. Progressive Education (8 numbers), Bulletins. 
Secondary Education Board. 
* Southern Association of Colleges and Secondary Schools, Proceed- 
ings. 
Southern Commission on Higher Institutions. 


American foundations and boards. Among the American 
foundations and boards which have promoted educational in- 
quiry through fellowships, exchange lectureships, encourage- 
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ment of research, subsidies granted to investigators, and 
through their own studies, annual reports, and publications 
are those listed below. The annual directory of the Office of 
Education includes the names and addresses of officers “and 
the place and date of meeting. The annual reports of these 
organizations include accounts of work completed and under 
way. Other surveys of the activities of boards and foundations 
are available.” 


American Field Service Fellowships for French Universities, 15 Broad 
Street, New York City. 

Anna T. Jeanes Fund, 726 Jackson Place N.W., Washington, D. C. 

Baron de Hirsch Fund, 730 Fifth Avenue, New York City. 

Carnegie Corporation of New York, 522 Fifth Avenue, New York 
City. : 

Carnegie Foundation for the Advancement of Teaching, 522 Fifth Ave- 
nue, New York City. 

Commission for Relief in Belgium Educational Foundation, 42 Broad- 
way, New York City. 

Commonwealth Fund, 41 East Fifty-seventh Street, New York City. 

Engineering Economics Foundation, 3 Joy Street, Boston, Massachu- 
setts, 

General Education Board, 61 Broadway, New York City. 

Harmon Foundation, 140 Nassau Street, New York City. 

John F. Slater Fund, 726 Jackson Place, N.W., Washington, D. C. 

John Simon Guggenheim Memorial Foundation, 120 Broadway, New 
York City. 

Josiah Macy Junior Foundation, 565 Park Avenue, New York City. 

Julius Rosenwald Fund, 4901 Ellis Avenue, Chicago, Illinois. 

Kahn Foundation for the Foreign Travel of American Teachers, 270 
Madison Avenue, New York City. 

Milbank Memorial Fund, 40 Wall Street, New York City. 

Payne Fund, 1 Madison Avenue, New York City. 

Phelps-Stokes Fund, ror Park Avenue, New York City. 

Rockefeller Foundation, 49 West Forty-ninth Street, New York City. 

Russell Sage Foundation, 130 East Twenty-second Street, New York 
City. 

Spelman Fund, 49 West Forty-ninth Street, New York City. 

63 A Handbook of Educational Associations and Foundations in the United States. 
Bureau of Education Bulletin, No. 16, 1926. Washington: Bureau of Education, Pp. 
ii Ee R. Evans, Educational Boards and Foundations, 1926-1928. Bureau of Educa- 
tion Bulletin, No. 9, 1929. Washington: Bureau of Education. Pp. 12. 


Pedro E. Y. Rio, Thirteen Educational Foundations and American Higher Education. 
Philadelphia: Temple University, r92s- Po. 108. 
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International educational associations and foundations. The 
student of education and field worker should recognize that 
international, as well as national and regional associations, 
societies, and foundations, are at work in the advancement of 
the science and practice of education. Evidence of interna- 
tional interest in education and of intellectual coöperation is 
found in such organizations, in numerous exchange scholar- 
ships and professorships, and in the granting oi funds for 
research on an international basis. Current periodical litera- 
ture also provides tangible evidence of this interest. For the 
selected organizations listed below, the names and addresses 
of officers, and place and date of meeting may be found in 
the annual educational directory of the United States Office 
of Education, Where permanent addresses are available, they 
are given. 


Institute of International Education, 2 West Forty-fifth Street, New 
York City. 

International Bureau of Education, 44 Rue des Maraichers, Geneva. 
Switzerland. 

International Council of Religious Education, 203 N. Wabash Avenue, 
Chicago, Illinois. 

International Education Board, 49 West Forty-ninth Street, New York 
City. 

World A for Adult Education, 16 Russell Square, Tondon, 
W.C. 1, England. 

World Federation of Education Associations, Galata Post Box 392. 
Istanbul, Turkey. 


Series of professional textbooks, public-school textbooks, 
and book lists. Certain publishing houses issue professional 
books in series, such as the Riverside Textbooks in Education 
(Houghton Mifflin), and the Appleton-Century Series in Su- 
pervision and Teaching. Some workers find it helpful to exam- 
ine the titles of volumes in such series when in search of in- 
formation on a given educational problem. A list of these 
series in education has been made available.“ The Education 


64 Frederick L. Whitney and Farle U. Rugg, “Sources of Current Literature in 
Education,” Teachers Journal and Abstract, I (January, 1926), 47-53- 
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Index publishes the titles of such series as are indexed in its 
columns. 

Book reviews and lists of current publications, published in 
professional journals, are helpful in locating new titles whether 
members of an educational series or individual volumes. 

The Society for Curriculum Study has initiated the publi- 
cation of a list of textbooks for the elementary and secondary 
levels. The list for 1931-33 includes 120 elementary and 1057 
secondary and junior-college texts, classified by school sub- 
jects. The total number of such textbooks published in 1934 
was 460. 

The American Educational Catalogue, the Cumulative Book 
Index, and the Publishers’ Weekly are also useful in securing 
a list of recent textbooks for public schools, with the United 
States Catalogue available for somewhat older books. 

The Education Index is valuable in locating book lists deal- 
ing with a variety of educational topics. An annual publication 
of the American Library Association is especially complete ; it 
gives all the lists of the association up to its date of publica- 
tion."* 

Alexander has provided a useful and somewhat detailed dis- 
cussion of how book lists and textbooks may be located.% 

Educational reference works. Although encyclopedias and 
more specialized reference works in education and psychology 
are of value, especially to those without advanced training or 
without other adequate library facilities, it must be recognized 
that the service rendered by such handbooks to the advanced 
student and research worker is limited. The expense of fre- 
quent revision and the brevity of treatment caused by limited 
space are problems faced by all editors and publishers, as well 


65 M. E. Herriott, “Elementary Textbooks of 1931-1933,” and “Secondary and Junior 
College Textbooks of 1931-1933,” Society for Curriculum Study News Bulletin, V 
(February 12 and April 9, 1934), 1-16, 1-76. 
and Others, “Textbooks of 1934,” Society for Curriculum Study News Bulletin, 
VI (January 15, 1935), 9-46. 
and Others, “Textbooks of 1935,” Curriculum Journal, VIL (February, 1936), 


15-43. 

66 American Library Association, Books and Pamphlets on Library Work. Chicago: 
American Library Association. Published annually. 

67 Carter Alexander, 9. cit., pp. 183-6, 216-9, 
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as users, of such reference volumes. A revision of Monroe’s 
encyclopedia would be especially welcome, although a very 
large undertaking. Among these works are: Paul Monroe’s 
An Encyclopedia of Education, F. Watson’s The Encyclopedia 
and Dictionary of Education, J. M. Baldwin’s A Dictionary 
of Philosophy and Psychology, H. C. Warren’s Dictionary of 
Psychology (1934), The Classroom Teacher (Chicago) , 
Mudge’s Guide to Reference Books, and the general reference 
works and encyclopedias such as the Britannica and Inter- 
national. 

A more complete list of dictionaries, encyclopedias, and 
related publications in education and psychology, including 
those published in Germany, Austria, France, and England 
follows: 


Unten STATES 


H. Kiddle and A. J. Schem, Cyclopedia of Education, 1877, first in the 
English language. 

James M. Baldwin, Dictionary of Philosophy and Psychology, 1901, 
three volumes; revised in 1928, Macmillan. 

Paul Monroe, Cyclopedia of Education, 1911-13, five volumes, Mac- 
millan. 

M. B. Hillegas,. Classroom Teacher, 1927-28, thirteen volumes. 

H. C. Warren, Dictionary of Psychology, 1934, Houghton Mifflin; de- 
fines ten thousand terms. 

H. B. English, A Student’s Dictionary of Psychological Terms, 1929, 
Antioch College. 

M. E. Haggerty, Topics in Psychology, 1929, University of Minnesota, 
2584 items. 

C. W. Odell, A Glossary of Three Hundred Terms Used in Educational 
Measurement and Research, 1928, University of Illinois. 

D. A. Robertson, Standard Terminology in Education, 1927, American 
Council on Education. 

L. Belle Voegelein, List of Educational Subject Headings, 1928, Ohio 
State University. 


EDUCATIONAL ENCYCLOPEDIAS IN ForEIGN COUNTRIES 


Germany 

I. G. C. Wörle, 1835. 

K. A. Schmid, 1876-87, ten volumes, 
W. Rein, 1903-10, ten volumes. 
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Austria 

Joseph Loos, 1906, two volumes. 
France 

F. Buisson, I9I1. 

England 


A. E. Fletcher, a one-volume handbook. 


Foster Watson, The Encyclopedia and Dictionary of Education, 1927, 
four volumes, Pitman. 


Graduate theses in education. Suggestions have been made 
earlier in this chapter for canvassing systematically the re- 
lated studies in the form of Masters’ and Doctors’ theses. For 
the sake of emphasis, even at the risk of some repetition, a 
fuller statement of the canvass of theses on related problems is 
made at this tirne. Like most other fields, the graduate thesis in 
education does not ordinarily have a sale sufficiently large to 
allow publication and distribution on a commercial basis. Ex- 
ceptions to this statement may be found in the monograph ser- 
ies of such institutions as Columbia and Harvard, although 
universities which publish the Doctor’s thesis usually require 
the student to underwrite the cost of publication. However, 
the majority of doctorate studies are unpublished, except for a 
brief abstract, and distribution of studies printed in full or in 
part is quite limited. Few Masters’ theses are printed except 
as summaries appear in educational journals or in volumes of 
graduate abstracts. Therefore, it will be recognized that, ex- 
cept for the worker who visits the library of the university 
where a given degree was awarded, most graduate theses are 
comparatively inaccessible. This condition is being remedied in 
part by interlibrary loans of theses and by the collection of 
graduate studies assembled by the Office of Education at 
Washington for lending purposes. The Office distributes occa- 
sional lists of its available theses and annual lists of all 
theses completed in education. In view of such difficulties, 
many graduate schools publish lists of theses completed or 
abstract volumes summarizing such studies. Some institutions 
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provide lists of graduate investigations under way in order to 
reveal centers of interest, promote a measure of coérdination in 
research, and avoid unnecessary duplication. The United States 
Library of Congress has an annual list of printed American 
doctoral studies, including all fields, beginning with 1912.°° In 
certain fields representative journals list annually thesis titles 
under way or completed in the area. represented, which is of 
value to investigators and field workers. 

Among the institutions represented in the chapter bibliog- 
raphy, which have compiled lists of theses completed or 
abstracts of such studies are California (Joyal), a group of 
California universities (Touton), Chicago, Cincinnati (Good), 
Illinois (Gregg and Hamilton), Indiana State Teachers College, 
Towa, Kentucky, Michigan, Minnesota (Engelhardt and Otto, 
Grinnell, Umstattd, Fink), Pennsylvania State College (Peters 
and Struck), Pittsburgh (Buckner), T emple, and Texas insti- 
tutions (Eby and Frost). 

Bibliographies or abstracts of theses in special fields are 
represented in the chapter bibliography as follows: secondary 
school administration (Douglass, Eikenberry, Reavis and 
Butsch, Roemer, Touton), sociology, chemistry (West and 
Hull), science (West and Hull), modern foreign language 
(Doyle), business education (Haynes and Graham), physical 
education (Browne), and Negro education (Knox). 

Other available information deals with Doctors’ theses under 
way in education (Good), Doctors’ theses completed over a 
ten-year period (Monroe), and theses on file with the Office 
of Education for lending purposes. Earlier in this chapter refer- 
ence was made to two other helpful sources for locating gradu- 
ate theses and dissertations, with the first reference devoting 
some space to interlibrary loans."° 

68 United States Library of Congress, A List of American Doctoral Dissertations 
Printed in 1931. Washington: Government Printing Office, 1933. See other lists 

inted in 1931 WGilchtist, Doctoral Dissertations Accepted by American Universities, 
7933-34. Number 1. New York: H. W. Wilson Co., 1934. Pp. 98. See later numbers. 

&9 Carter Alexander, op. cit., pp. 152-9, 223-9. 


Clara E. Derring, “Lists and Abstracts of Masters’ and Doctors’ Dissertations in Edw 
cation,” Teachers College Record, XXXIV (March, 1933), 490-502. 
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Leaders in education and psychology, and educational di- 
rectories. Most of the preceding portion of this chapter has 
been concerned with the various types of educational literature 
available. It seems desirable to devote some space to the insti- 
tutions, agencies, leaders, authors, and investigators who pre- 
pare or sponsor such information. The characteristics and 
training of the “man behind the gun” are important matters in 
military organizations and operations. No less significant for the 
field of research are the characteristics, training, distribution, 
activities, positions, institutional affiliations, etc., of those who 
conduct scientific investigations and produce educational lit- 
erature. This problem is discussed at some length in Chapter 
XV in terms of the training and supervision of research work- 
ers; however, it seems appropriate at this time to enumerate 
the various guides supplied by biographical directories; analy- 
ses of the characteristics and activities of educators, scientists, 
and psychologists; and directories and statistics of the institu- 
tions with which such workers are affiliated. 

Biographical directories. The directories which provide bio- 
graphical data concerning prominent workers in education, 
psychology, and allied scientific fields are: Who’s Who, Who’s 
Who Among North American Authors, Who’s Who in America, 
Who’s Who in American Education, Psychological Register, 
Leaders in Education, American Men of Science, and the Phi 
Delta Kappa Directory (publication of a national professional 
fraternity in education.) Other biographical directories for se- | 
curing information about individuals are listed in the chapter 
bibliography. More detailed guidance for this type of library 
procedure has been provided by Alexander, who describes 
briefly the various biographical directories.” 

Although few biographical data are given in the membership 
lists of professional organizations (such as the National Society 
for the Study of Education, National Society of College Teach- 
ers of Education, Sections Q and I of the American Association 
for the Advancement of Science, American Educational Research 


z0 Carter Alexander, op. cit., pp. 230-42, 
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Association, Department of Superintendence, Department 
of Elementary-School Principals, Department of Secondary- 
School Principals, Department of Supervisors and Directors 
of Instruction, and the American Psychological Associa- 
tion), frequently the limited information provided is sufficient 
to meet a given need. In most instances these lists appear in 
a year-book, journal, or other official organ. Obviously, it is in- 
advisable in the limited space available to compile from these 
directories a list of workers prominent in the different fields 
of educational writing and research. The various studies men- 
tioned in the bibliographies of this book are a fair indication 
of the individuals most active in the several phases of educa- 
tional research. 

Likewise, it seems impracticable to suggest a list of contribu- 
tions made by individual workers in the various fields, or a 
statement of current research projects. By the time this chap- 
ter reaches the reader, such a list of contributions would not 
be complete and the research projects no longer would be 
contemporary. The intellectual activity and growth of the uni- 
versity student and worker in service will be stimulated more 
effectively, and his individual interests met more adequately, 
if he performs these services for himself. The type of informa- 
tion outlined in the present chapter makes this canvass pos- 
sible with a minimum of waste motion. 

It will be recognized that the nature of the contribution made 
varies with the individual worker. An editorial writer states 
that there are two types of Jeaders in education—innovators 
who establish “experimental” schools where traditional sub- 
jects are abolished and centers of interest set up, and scientists 
who search for facts on which to base curriculum materials 
and new procedures in promoting the learning process. Cer- 
tainly the progressive student of education and teacher in 
service will desire to become acquainted with the contribu- 
tions of both types of educational workers. Another educational 
editor suggests that there are three kinds of leaders who make 


71 “Invention and Appraisal,” Educational Method, 1X (December, 1929), 129-30- 
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valuable contributions—the administrator, the investigator, 
and the interpreter.*? 

Attempts to compile selected lists of educational writers and 
workers. Although it seems inadvisable to present a list of 
frequent contributors to the literature of educational research, 
it is of interest to note the activities considered in two attempts 
to compile such a selected group. One of the authors’ classes 
used Plan I in compiling a list of 168 contributors to educa- 
tional literature of a factual nature. Miss Jane Adams, at Stan- 
ford University, used Plan II in choosing one hundred eminent 
educators. The amount of overlapping in the two resulting 
groups of workers is evidenced by the fact that seventy of the 
one hundred men on the Adams list were represented in the 
group of 168 discovered by Plan I. 

Of course, the shortcomings of such attempts to identify edu- 
cational leaders, caused by the sins of omission and com- 
mission, are evident without extended comment. It will be 
recognized that mere frequency of publication and number of 
years served in a particular type of position do not indicate 
relative values in terms of the contribution made to education. 
Such books as Dewey’s Democracy and Education and The 
School and Society have had more influence on educational 
philosophy than hundreds of certain other published volumes, 
The relatively short period of service permitted to Inglis was 
of more significance to secondary education than the work and 
writing of many others of less vision and insight. At any rate 
the check-lists reproduced below do Suggest useful activities, 
experiences, and areas of service in the field of educational en- 
deavor. 


Plan I7 


1. Monroe, Ten Years of Educational Research, 1918-1927 (Authors 
with three or more references to their credit, two points for each 
reference.) 


72J. E. Morgan, ‘Educational Jnterretation and the Printed Word,” School ang 
Society, XXX (July 20, 1929), 79-83. 
78 Used by a class in Thesis Writing and Research at the University of Cincinnati. 


» 


we 
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List of Sixty Educational Books, Journal of the National Educa- 
tion Association, for past five or more years (Ten points for each 
title.) 


Professional year-books and proceedings (One point for work as 
a committee member ox contributor to a chapter.) 

Classroom Teachers 

Elementary-School Principals 

Secondary-School Principals 

Department of Superintendence 

National Education Association Proceedings 

American Association of Teachers Colleges 

National Conference of Supervisors and Directors of Instruc- 

tion 
National Society for the Study of Education 
National Society of College Teachers of Education 


Teachers Journal and Abstract, from date of first appearance 
(One point for-each article.) 


Plan II ™* 


Inclusion of name in 1930-31 Who’s Who in America (All ma- 
terial for checking was secured from the individual’s biography in 
Who’s Who except the score on magazine articles which was the 
result of checking the Readers’ Guide for the years 1920 to 1930 
inclusive.) 


Ratings: Points 

1. Magazine articles (educational) 1 each 
(Estimate allowance to be made where there is a 
large number of one-page articles or articles of ob- 
viously minor importance, or department or official 
reports not distinctly of author’s composition.) 

2. Books on education 4 each 
Co-author—2 points 
New edition—2 points 


Educational articles in encyclopedias 1 each 
3. Editor of educational books 1 each 
4. Editor of educational series of books 2 each 
5. Editor of educational journal (for each five-year 
period) I each 
6. Writing in other fields 
7. Educational surveys made (as director or member i 
I eac 


of committee) 


74 Used by Jane Adams at Stanford University. 
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8. Educational or intelligence tests each 
9. Service on educational committee I each 


a 


to. War service (educational) 1 total 
11. Highest degree earned 
Ph.D. or Ed.D. —2 
M.A. —1% 
B.A. —1I 
12. Honorary degrees I each 
13. Member, or officer, of professional educational as- 
sociation I or 2 
— total 
14. Member, or officer, of Phi Beta Kappa or any hon- 
orary educational fraternity I or 2 
— total 
15. Positions held: 
Elementary or high school A 
Normal school or college I 
College president 3 
University professor 14 


Some qualifications, where rating not equable, as, 
university professor: 


I year only % 
2—5 years I 
ro or more 2 
20 or more 4 
Dean, school of education 3 
President of university 4 
City or county superintendent H 
State superintendent of schools 3 
United States Commissioner of Education 4 
United States Office of Education I 
Summer school (college or university) A 


It is of more than passing interest, as well as of informa- 
tional value, to note the manner in which certain educational 
leaders with national and international reputations have been 
recognized by their institutions, fields, or students, or by com- 
pilations of lists of their publications: celebration of the sev- 
entieth birthday of John Dewey,’> commemoration of the 

75 “In Honor of John Dewey,” Teachers College Record, XXXI (December, 1929), 
n Dewey lee of the National Education Association, XVIII (December, 
1929), 281-96. 


M. H. Thomas and H. W. Schneider, A Bibliography of John Dewey, New York: 
Columbia University Press. 1929. Pp. 172. 
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twenty-fifth anniversary of Thorndike’s appointment to the 
staff of Teachers College of Columbia University, recognition 
of the contributions of Charles H. Judd together with a list of 
his publications,” a review of the work of the late B. T. Bald- 
win," a series of articles on the life and work of E. P. Cub- 
berley and a book in his honor,” the W. T. Harris Centennial,*° 
a quarter of a century of service by Paul Monroe,** the writ- 
ings ** of Nicholas Murray Butler, and the history of psy- 
chology in autobiography.** Two current attempts to analyze 
the social attitudes and ideas of leaders in American education 
have attracted a great deal of attention and are well worth 
the attention of the critical reader.** 

Although the reader will doubtless agree that it is undesir- 
able for authors or students of research to compile an extended 
or purportedly complete list of scientists, psychologists, or edu- 
cational leaders, some profit by way of intellectual stimula- 
tion may accrue to the beginner in graduate work through 
scanning lists of the presidents of two of the most scholarly 
American organizations, the American Association for the Ad- 
vancement of Science and the American Psychological Associa- 
tion. The presidents of the latter organization are listed in the 
annual directory (year-book) of its members. 


76 “Annotated Chronological Bibliography of Publications by E. L. Thorndike,” 
Teachers College Record, XXVII (February, 1926), 466-515. 

77 “In Honor of Charles Hubbard Judd,” Zeta News (Phi Delta Kappa, University 
of Chicago), XII (April, 1927), 1-33. 

78 Dorothy E. Bradbury and Marion Hossfeld, “A Review of the Published Writings 
of Bird Thomas Baldwin,” Psychological Bulletin, XXVIII (April, 1931), 257-76. 

79 “Cubberley (A Series of Articles on His Life and Work),” California Quarterly of 
Secondary Education, VIII (April, 1933), 229-49. 

Also see the commemorative volume of papers by J. C. Almack and Others, Modern 
School Administration, Boston: Houghton Mifflin Co., 1933. Pp. 382. 

80 “William Torrey Harris,” International Education Review, IV (1935), 235-316. 
A series of articles on the contributions of Harris. 

811. L. Kandel, Editor, Twenty-five years of American Education: Collected Essays. 
New York: The Macmillan Co., 1924. Pp. xvi + 469. 

82M. H. Thomas, Compiler, Bibliography of Nicholas Murray Butler, 1872-1932. 
New York: Columbia University Press, 1934. Pp. 438. 

83 Carl Murchison, Editor, A History of Psychology in Autobiography, Vols. 1, II. 
Worcester, Mass.: Clark University Press, 1930, 1932. Pp. 516, 407. 

84 Norman Woelfel, Molders of the American Mind: A Critical Review of the Social 
Attitudes of Seventeen Leaders in American Education. New York: Columbia University 
Press, 1933. Pp. 304. 

Merle Curti, The Social Ideas of American Educators. Part X, Report of the Com- 
mission on the Social Studies, American Historical Association. New York: Charles 
Scribner’s Sons, 1935. Pp. xxiv + 613. 
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Analyses of the activities and characteristics of educators, 
scientists, and psychologists. The present topic as related to 
the training of research workers is discussed in Chapter XV. 
The casual student and school worker hardly realize the 
amount of space given to analyses of the activities and traits of 
educators and scientists in educational literature. This state- 
ment may be substantiated by a number of examples. Odell 
analyzed twenty-one sources to note the frequency of mention 
of workers in educational measurement and ranked the first 
twenty-five investigators in this field. Shannon made a study 
of the contributors to eight educational journals from 1926-30 
inclusive,** while Cowley and Hankins respectively analyzed 
student personnel literature and sociological publications in 
terms of authorship.’’ Wiley reports a good relationship be- 
tween the extent of educational writing done and salaries re- 
ceived, although a direct causal relationship may not obtain; 
both may be concomitant results of professional competence.** 

Cope presents data on marital state, age at marriage, 
fecundity, education, etc., of the women in Who’s Who. 
Pressey reports information concerning birthplace, age, degrees, 
major feld of interest, occupation, and teaching location of the 
women whose names appear in American Men of Science,” 
while Poffenberger studied the men in this group.” Cattell 
shows the distribution of American men of science, and Visher 
tabulates the institutions from which “starred” scientists first 
graduated, as found in the biographical directory of men of 

85C. W. Odell, “Who Have Contributed Most to the Educational Measurement 
Movement?” School and Society, XXIX (June 8, 1929), 751-4. 

86 J. R. Shannon, “Contributors to Educational Periodicals,” Elementary School 
Journal, XXXII (November, 1931), 177-82. 

87 W. H. Cowley, “Who Produces Student Personnel Literature?” School and Society, 
XXV (May 7, 1932), 638-40. 

Dorothy Hankins, “Living Leaders in Scientific Sociology,” Sociology and Social Re- 
search, XV (September-October, 1930), 37-46. 

88 Roy W. Wiley, “Educational Writers and Their Salaries,” University of Pittsburgh 
School of Education Journal, IV (January-February, 1929), 61-7. 

89 Persis M. Cope, “The Women of ‘Who’s Who’: A Statistical Study,” Social Forces, 
VII (December, 1928), 212-23. 

90 Luella C. Pressey, “The Women Whose Names Appear in ‘American Men of 
Science,” School and Society, XXIX (January 19, 1929), 96-100. 


91 A. T. Poffenberger, “The Development of Men of Science,” Journal of Social 
Psychology, I (February, 1930), 31-47- 
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science.*? Lehman and Witty’s analysis of scientific eminence 
and church membership is based on data from Who’s Who in 
America and American Men of Science.® A study of the pro- 
fessional experience of professors of education and of the 
geographical distribution of leaders in education is based on in- 
formation in Leaders in Education.* Greene’s study of the 
education possessed by Negro leaders involves information se- 
cured from biographical directories of national scope.*® 

The college presidency, as a position of educational leader- 
ship, has been analyzed frequently. In fact, a biographical 
directory of the presidents of higher institutions is now avail- 
able. The contribution of higher institutions to educational 
leadership has been analyzed in terms of the number of repre- 
sentatives from various colleges appearing in Who’s Who in 
America.” 

Educational positions of some importance, involving oppor- 
tunities for leadership, have been analyzed in terms of the vo- 
cational paths pursued by 546 persons before reaching such po- 
sitions.’ The degrees and positions held by 8,138 members of 
Phi Delta Kappa fraternity, a professional organization in 
education, have been tabulated and reported.” Rainey presents 


92 J. M. Cattell, “The Distribution of American Men of Science in 1932,” Science, 
LXXVII (March 10, 1933), 264-70. 

S. S. Visher, “Starred Scientists,” Journal of Higher Education, II (February, 1931), 
78-80. 

‘Also see Wilson Gee, Research Barriers in the South. New York: The Century Co., 
1932. Pp. x+ 192. Includes an analysis of southern-born social scientists found in 
American Men of Science and Who's Who in America. 

93H. C. Lehman and P, A. Witty, “Scientific Eminence and Church Membership,” 
Scientific Monthly, XXXIII (December, 1931), 544-9. 

94 J. R. Shannon, “Professional Experience of Professors of Education,” School and 
Society, XXXVI (November 12, 1932), 638-40. } 
“Geographical Distribution of Leaders in Education,” School and Society, 
XXXVI (December 3, 1932), 720-2. $ 5 

95 H. W. Greene, “The Education of Negro Leaders,” School and Society, XLII (July 
27, 1935), 134-6. A aah? 

96 R, C. Cook, Presidents of American Colleges and Universities. New York: Robert 
C. Cook Co., 1933. Pp. 256. ae j 

97D. B. Prentice and B, W. Kunkel, “The Colleges’ Contributions to Intellectual 
Leadership,” School and Society, XXXII (November 1, 1930), 594-600; XXXIII 

Febru » 1931), 280-2. y Sa 
eR Ree Payocational Paths to Certain Educational Positions,” School and 
Society, XXXVI (October 15, 1932), 508-12. 4 

99 A E. Traxler, “The Educational Status of Phi Delta Kappans.” Phi Delta Kappan, 


XIV (August, 1031). 50. 
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facts concerning 192 presidents of colleges belonging to the 
Association of American Universities, the North Central Asso- 
ciation of Colleges and Secondary Schools, and the Southern 
Association of Colleges and Secondary Schools, as gleaned from 
Who's Who in America Palmer discusses fifty-five changes 
in college presidencies and gives information concerning the ex- 
perience, vital statistics, motives in acceptance, etc., of these 
presidents.‘ Robinson bases certain conclusions on an analysis 
of the sex of college presidents and on statements from promi- 
nent educators.1°? 

Psychologists and their laboratories have also received a 
considerable amount of attention in current periodical litera- 
ture. Fernberger gives a cross section of the distribution and 
publications of members of the American Psychological Asso- 
ciation.* Ching-Ju Ho includes an analysis of the vocational 
histories of prominent psychologists in his personnel studies of 
scientists. Cattell discusses the distribution and organization 
of psychologists in America,1% Garvey lists for 117 institutions 
information concerning the date of the first psychology labora- 
tory, first director, amount of original grant, date of first pub- 
lication, and author. Cattell 1°? and Fernberger #8 discuss 


100 H, P, Rainey, “Some Facts about College Presidents,” School and Society, XXX 
(October 26, 1929), 580-4. 

101 A. M. Palmer, “Newly Elected College Presidents,” School and Society, XXX. 
(December 21, 1929), 852-6. 

102 F, B. Robinson, “Women as Prospective College Presidents,” School and Society, 
XXX (August 24, 1929), 245-52. 

103 Samuel W. Fernberger, “Statistical Analyses of the Members and Associates of 
the American Psychological Association, Inc., in 1928: A Cross Section of American 
Professional Psychology,” Psychological Review, XXXV (November, 1928), 447-65. 
» “The Publications of American Psychologists,” Psychological Review, 
XXXVII (November, 1930), 526-44. 

V. A. Jones, “Fields of Instruction and Research in Psychology as Represented by 
Members of the American Psychological Association,” Journal of General Psychology, 
X (January, 1934), 211-4. 

104 Ching-Ju Ho, Personnel Studies of Scientists in the United States, Teachers College 
Contributions to Education, No. 298. New York: Teachers College, Columbia University, 
1928. Pp. 60. 

105 J. M. Cattell, “Psychology in America,” Science, LXX (October rx, 1929), 335-473 
Scientific Monthly, XXX (February, 1930), 114-26. 

106 C. R. Garvey, “List of American Psychology Laboratories,” Psychological Bulletin, 
XXVI (November, 1929), 652-60. 

107 J, M. Cattell, “Early Psychological Laboratories,” Science, LXVII (June 1, 1928), 
543-8. 

108 S. W. Fernberger, “The American Psychological Association: A Historical Summary, 
1892-1930,” Psychological Bulletin, XXIX (January, 1932), 1-89. 
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respectively early psychological laboratories and the growth of 
the American Psychological Association. 

Information is also available concerning the foreign-language 
training of holders of the Ph.D. degree, methods of training 
great teachers, and the longevity of educators. Betts and Kent 
describe the foreign-language equipment of 2,325 Doctors of 
Philosophy ; naturally it is expected that the individuals of this 
group will furnish much of the educational leadership in their 
respective fields.1 Kelly and Anderson inquired of 349 repre- 
sentative college and university executive officers concerning 
methods of producing great teachers."*° Luckey reports data 
concerning the length of life of eminent educators in Greece, 
Italy, Scotland, Switzerland, France, Germany, England, and 
the United States.™ 

Directories and statistics of educational institutions. Much 
significant information is to be found in the educational direc- 
tories of accrediting organizations, of state and city school 
systems, and of higher institutions. The United States Office 
of Education publishes an annual educational directory which 
includes such items of information as principal state school 
officers, county superintendents, superintendents of public 
schools, superintendents of Catholic parochial schools, univer- 
sities and colleges, presidents or deans of professional schools, 
educational boards and foundations, and educational and 
learned associations, as well as a list of other available educa- 
tional directories. The Office of Education also publishes lists 
of accredited secondary schools and higher institutions, and 
statistical surveys of the school system at all levels. Patter- 
son’s American Educational Directory is a useful handbook. 
Other directories, handbooks, or statistical surveys represented 
in the chapter bibliography deal with universities and colleges 


109 G. H. Betts and R. A. Kent, Foreign Language Equipment of 2,325 Doctots of 
Philosophy. Northwestern University Contributions to Education, No. 2. Bloomington, 
Il.: Public School Publishing Co., 1929. Pp. 152. 

110 Robert L. Kelly and Ruth E. Anderson, “Great Teachers and Some Methods of 
Producing Them,” Journal of Educational Research, XX (June, 1929), 22-30. 

111 G. W. A. Luckey, “Longevity of Eminent Educators,” School and Society, XXVIII 


(August 25, 1928), 244-8. 
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(Halle, Hurt, MacCracken, Robertson, Walters), junior col- 
leges (Campbell, Eells), Christian education (Kelly and An- 
derson), Catholic colleges and schools, graduate schools 
(Hughes), presidents of American colleges and universities, 
private-school teachers, private schools for American boys 
and girls, etc. The chapter bibliography should be scanned rap- 
idly to locate the particular type of handbook or directory 
desired. Detailed directions for use of such reference volumes 
in collecting information about individuals or institutions are 
available in another source.*” 

Another chapter (XI) of this book deals with the statistical 
concepts essential to correct interpretation of educational data 
and statistics. Tyler has recommended a desirable content for 
courses designed to give training for intelligent reading of sta- 
tistical literature." The range in items of information collected 
by ten different state departments of education has been an- 
alyzed by Reavis to show what types of facts are available in 
the states in question. General volumes of statistics listed in 
the chapter bibliography, which cover a wide range of topics, 
including education, are the Statistical Abstract of the United 
States, the World Almanac, and the Statesman’s Yearbook. 

News items, quotations, and pictures. There are many times 
when the educational worker has occasion to search for news 
items, quotations, proverbs, pictures, illustrations, and por- 
traits. An excellent discussion of library procedure in locating 
such information has been provided,“ and in the chapter 
bibliography a number of valuable guides for use in this type 
of work are listed. 

Systematic organization of educational literature and in- 
formation. The school worker who sees the problems of educa- 


112 Carter Alexander, How to Locate Educational Information and Data, op. cit, 
PP. 187-99, 230-42. 

118 R. W. Tyler, “Statistics Needed by Readers,” Educational Research Bulletin, IX 
(April 16, 1930), 205-11. 

114 W, C. Reavis, “Items of Information Collected by Departments of Public Instruc- 
tion of Ten Representative States,” Elementary Sckool Journal, XXIX (May, 1929), 

6-73. 

115 Carter Alexander, op. cit., pp. 243-62 
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tion as component parts of a unified whole, rather than in isola- 
tion, promises most in the solution of such difficulties, A similar 
recognition of relationships is helpful in working with the lit- 
erature of educational investigation. Some workers have found 
it worth while at intervals, in maintaining perspective, to re- 
view an outline of the entire range of educational problems at 
all levels—elementary, secondary, and higher. (For example, 
see the analyses in the appendix.) 

A rather detailed statement of the problems of secondary 
education is presented in the accompanying outline as an illus- 
tration of the way in which each major level or field of educa- 
tion may be subdivided. The literature of secondary education 
ialls rather naturally under such headings. It may be sug- 
gested that the worker in secondary education will find it con- 
venient and profitable to group educational information accord- 
ing to this, or a similar, organization of topics of interest. It 
is entirely possible that many workers may prefer to organize 
materials along functional lines, such as administration, super- 
vision, curriculum, psychology, etc., rather than by levels of 
instruction, As has been suggested, secondary education has 
been used here only as an incomplete example of what may be 
done by way of classification and subdivision. Attention 
already has been directed to the importance of subject head- 


ings in library work. 


Divisions OF THE FIELD or SECONDARY EDUCATION 


I. Secondary Administration and Supervision 
1. Educational administration, organization, and management 
Business administration, finance, buildings and grounds 
Schcol legislation 
School surveys 
Extracurricular activities 
Educational and vocational guidance 
Marks and examinations 
Pupil grouping 
Schedule-making 
Records 
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II. 
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Size of class 
Student government 
Relation to community and higher institutions 


II. Psychology of the Secondary School and Measurement 


I. 


PONT ONCE Ce 


9. 
I0. 


Psychological tests 

Educational tests and measurements 

Statistical methods 

Psychology of learning 

Adolescent psychology 

Individual differences 

Education of exceptional children 

Experimental education, research, and the scientific move- 
ment 

Psychology of the school subjects 

Failure and elimination, diagnostic and remedial work 


III. General Theory and General Methods of Teaching 


I. 
2. 


3. 


4. 
5 


6. 


7. 


8 


Aims and objectives 

Selection and organization of subject-matter 

Economical bases of learning: self-activity, motivation, in- 
terest, apperception, frame of mind, etc. 

Types of learning and teaching 


. Study problems: directed study, reading, how to study, 


use of books 

Miscellaneous methods and problems: project, problem, 
conversational and discussion, lecture, laboratory, ques- 
tioning, and visual methods 

Educational philosophy 

Educational sociology 


Iv. Secondary Curriculum and Special Method 


Commercial and business education 


2. English 

3. Fine arts 

4. Foreign languages 

5. Home economics 

6. Manual and industrial arts 

7. Mathematics 

8. Moral and character education 
9. Music 

to. Physical and health education 
Ir. Science 

12. Social studies 


V. History of Secondary Education 


I. 
2. 
3. 


Educational biography 
Current educational conditions 
Comparative secondary education 
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VI. Reorganization and Adjustment in the Secondary School 
I. Junior high school 
2. Junior college 
3. Curriculum 


Concluding statement. The competent physician must keep 
constantly abreast of the latest discoveries in the field of medi- 
cine, since human life itself is at stake. He knows the specialist 
to whom a difficult case should be referred for most effective 
treatment. Of course, the most skilled practitioner or surgeon 
cannot keep in mind all the details of his calling. However, he 
is adept in locating information promptly, when needed, in his 
case books and medical journals. The successful lawyer must 
be able readily to locate information pertinent to the case at 
hand in the records of court decisions and in the statutes. Cer- 
tainly those in charge of the intellectual and educational de- 
velopment of youth are under an equally great obligation to 
secure the best available information concerning the most ef- 
fective learning, teaching, and administrative procedures. Ob- 
viously the careful student of education, the research worker 
and investigator, the critica! supervisor and administrator, and 
the thoughtful classroom teacher should become familiar with 
the location and use of the sources of educational information 
listed in this chapter, even though he actually reads regularly 
and systematically only a small portion of the available ma- 
terial. 

When Thomas A. Edison was a boy he set out to read all 
the material in the Detroit Public Library, which of course 
he did not accomplish. However, during his long life, he did 
read everything available on the subjects with which he was 
working, as well as on many other apparently remote matters. 
His established procedure was to determine what others had 
done and then go forward from this point.® 

The bibliographical survey of related factual, experimental, 
theoretical, and historical materials orients the investigator and 
the problem in terms of the adequacy of the available evidence, 


116 T. A. Boyd, Research, p. 72. New York: D. Appleton-Century Co., 1935- 
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current ideas and hypotheses, and appropriate methods of re- 
search. (In this connection, the material of Chapter VI on the 
historical method should prove helpful.) F urthermore, it has 
been maintained that the library and its array of pertinent lit- 
erature are means of holding together a profession which tends 
more and more to subdivide and specialize, as in medicine and 
education." 

It has been well said that the scholarship of an institution or 
region depends on, and to a large extent can be measured by, 
the adequacy of its library facilities. 


PROBLEMS AND EXERCISES 


1. Select a topic in which you are especially interested, and specify 
in some detail the variety of library sources which you would 
examine in working out a reasonably complete preliminary survey 
of related studies and literature. 

2. Prepare a brief working bibliography on the topic selected in con- 
nection with the preceding problem, stressing the literature of the 
two years preceding the undertaking of this assignment. (See Chap- 
ter XIII for bibliographical form.) 
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CHAPTER IV 


THE FORMULATION AND TESTING OF HYPOTHESES 


Definition of a hypothesis. When the mind has before it a 
number of observed facts about some phenomenon which has 
attracted its attention for one reason or another, there is a 
tendency to form some generalization relative to the character 
of the phenomenon. These tentative generalizations formed 
about the character of phenomena under observation are called 
hypotheses. (A hypothesis is a statement temporarily accepted 
as true in the light of what is, at the time, known about a 
phenomenon, and it is employed as a basis for action in the 
search for new truth. When the hypothesis is fully established it 
may take the form of facts, principles, or theories.) Usually sound 
theories relative to complex phenomena are reached only after 
many hypotheses have been tried out and discarded or modi- 
fied to harmonize with established facts. Whether true or false, 
our present-day theories of learning have, for example, been de- 
rived only through the careful checking, discarding, and modi- 
fying of a long series of hypotheses. A hypothesis, therefore, is a 
guess, supposition, or tentative inference as to the existence of 
some fact, condition, or relationship relative to some phenome- 
non which serves to explain such facts as already are known to 
exist in a given area of research and to guide the search for 
new truth. 

Importance of hypotheses. It is sometimes proposed that 
one collect all of the facts about the objects of educational 
research without any preconceived idea about them. The diffi- 
culty with this conception is that one ordinarily sees only 
those things for which one looks and, unless the investigator 
approaches the study of the more complex phenomena of life 
with some ideas about the probable facts, conditions, and rela- 
tionships involved, he will not learn much about them. The 
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hypothesis thus serves the threefold function of limiting the field 
of investigation, of sensitizing the observer to pertinent facts, 
and of colligating the facts observed about different phenomena 
into some simple form. In the first place, if the field is not limited, 
much valuable time is wasted in aimless research. If one has lost 
or misplaced some object about the home, it is ordinarily much 
better to attempt to recall the circumstances surrounding the loss 
of the object, to restrict the area for search, than to search the 
entire house from cellar to attic each time some article is mis- 
placed. And so it is with the study of the problems of life and 
education. To limit the search to profitable areas of investiga- 
tion, the research worker must bring to each investigation ideas 
of the most probable facts, conditions, and relationships in- 
volved ; this procedure conserves valuable time in the search for 
truth. A second function served by the hypothesis is to sensitize 
the individual to facts, conditions, and relationships that might 
otherwise be passed by unheeded. It is a well-known psycho- 
logical fact that attention is selective and that the purpose of 
the investigator determines in a very real way the things ob- 
served. If one is, therefore, to observe the important ante- 
cedents of the objects of educational research, he must be sen- 
sitized to them by carefully chosen suppositions. Finally, the 
hypothesis serves the important function of offering a simple 
means of collecting large bodies of unorganized facts into one 
comprehensible whole. Like a good classification, a good hy- 
pothesis serves the important function of linking together re- 
lated facts and information and organizing them into wholes. 
To be most helpful, a hypothesis must serve these three func- 
tions. 

The use of hypotheses in different types of research. In a 
succeeding chapter of this book the methods of research are 
reduced to four major types: (1) historical research; (2) 
normative-survey research; (3) experimental research; and 
(4) other complex causal research. Each of these types of re- 
search may be made to serve different purposes: to furnish a 
reliable record of past events, to trace the sequence of events 
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by which some past or present observed condition may have 
arisen, to describe the status of some past or present phe- 
nomenon, to discover the antecedents of events in the pres- 
ent, etc. The use of hypotheses will vary somewhat with the type 
of research employed and the purpose to be served. In his- 
torical research the purpose may be either to produce a faithful 
record of the past, irrespective of present-day problems, or to 
extend the experiences with phenomena in the present to the 
past in order to make the view of the phenomena more com- 
plete. For the latter purpose, hypothetical thinking serves the 
conventional purposes of limiting the search, sensitizing the 
observer, and colligating facts. When the purpose, however, is 
to produce a faithful record of the past, irrespective of present 
felt needs, little or no use is made of hypothetical thinking 
except in establishing the record itself. In normative-survey 
research the investigator may or may not employ the hypo- 
thetical type of thinking, depending upon his purpose. If he 
desires merely to know the status of certain phenomena— 
for example, the age, training, and experience of elementary- 
school teachers—little or no use may be made of the hypo- 
thetical type of thinking. If, however, the investigator is 
working with some problem for which the solution depends 
upon study of the status of various diversified phenomena and 
their relationships—for example, the play interests of children 
of different nationalities, the relative cost of consolidated and 
nonconsolidated rural schools, or sex differences in the aca- 
demic achievement of secondary-school pupils—hypothetical 
thinking may play an important rôle, serving the more im- 
portant functions of this type of thinking. Either rightly or 
wrongly, hypothetical thinking does not appear to assume the 
significant rôle in normative-survey research that it does in 
either experimental or uncontrolled causal research, In these 
types of research hypothetical thinking is best developed and 
plays an important part.’ 


1R. H. Wheeler, “Postulates for a Theory of Education: II, A Methodology for 
Educational Research,” Journal of Educational Research, XXIX (November, 1935), 
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Experiment is not the main source of scientific principles. It is the 
means of testing a deduction. Insight and creative imagination, relative 
to observation, are the important sources of principles. 

Measurement is not the source of laws. Rather, it permits expres- 
sion of a previously conceived law in mathematical terms. A con- 
ception of a law is necessary before an experiment can be planned, 
apparatus constructed, and units of measurement decided upon. In 
other words, measurement depends upon the conception of laws. 


Two important aspects of hypotheses in thinking. There 
are two aspects of hypotheses, as used in thinking, that need 
to be considered: (1) the discovery of hypotheses and (2) the 
testing of hypotheses. These facts suggest three questions: (a) 
How does one secure ideas as to the most profitable areas of 
investigation? (b) What are the characteristics of a good 
hypothesis? and (c) How are the hypotheses tested? Each of 
these three questions will be considered in some detail in the 
remaining pages of this chapter. 

The discovery of hypotheses. One of the most important 
aspects of any investigation is that of securing sound ideas 
about the facts, conditions, and relationships involved in the 
phenomenon. This itself has two phases: one, that of getting 
ideas; and the other, that of securing good ideas. If ideas are 
not good, much valuable time and energy are wasted, and defi- 
nite harm may be done by delaying the correct solution to an 
important problem. How, then (and this problem needs careful 
consideration), does one secure sound ideas about the facts, 
conditions, and relationships involved in the problems of edu- 
cational research? 

The importance of being well-informed. Reference has 
been made earlier to a statement that the original mind is the 
well-informed mind. As a matter of fact, it would be difficult 
to over-emphasize the importance of being well informed about 
both the specific problem under investigation and related 
phenomena. Frequently, ideas of what to do in some particular 
situation (for example in studying the abilities of teachers), 
arise not so much out of one’s knowledge of the situation itself 
as out of information concerning related phenomena, The 
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science of chemistry, for example, has been built up around the 
ideas of the kinds, amounts, and arrangements of the constitu- 
ents of matter. Thus, there are qualitative chemistry, quan- 
titative chemistry, etc. This fact suggests the idea of qualitative 
and quantitative differences in the abilities of teachers. Again, 
one reads in the study of physics that many objects with very 
different exteriors may operate according to the same principle, 
and that the principle rather than the exterior is the important 
consideration. This suggests the idea of, and the importance of, 
principles of teaching, and that specific teacher and pupil 
activities, significant as they are, represent merely the super- 
ficial aspects of good teaching. Such theories of teaching arise 
not merely from one’s knowledge of teaching itself but from in- 
formation concerning a related science. 

For further evidence of the necessity of being well informed, 
one may turn to the events of ordinary life where frequently 
there is occasion to determine the causes or antecedents of 
phenomena of one sort or another: one’s car fails to operate 
satisfactorily, one is not feeling as well as one desires, or one’s 
fruit orchard is not producing as it should, etc. If one possesses 
the appropriate knowledge, the necessary adjustments may be 
exceedingly simple. Even if one must employ the services of a 
trained worker, there, too, the first essential to expertness is 
to be well informed. The expert mechanic, doctor, or agricultur- 
alist is one who is well informed about the antecedents of the 
ordinary instances of malperformance, and who has become 
skilled in locating these causal or conditioning factors through 
practice. In a similar fashion the expert educational diagnos- 
tician, whether in the laboratory or in the field, is the one who 
is exceedingly well informed. Although the primary function 
of the field worker is not to produce educational generalizations 
for general consumption, he must be a student of the educative 
process, and he must be well informed both in this capacity and 
as a practitioner. This point of view has been emphasized in 
Chapters IT and HI. 

The preceding reference to the practitioner calls to mind the 
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importance of first-hand contact with the objects of research. 
Many experienced research workers advise that investigators 
come into direct contact with the objects of study as soon as 
possible through some sort of a preliminary investigation, sur- 
vey, or experiment. The point of this suggestion arises out of 
the difficulty of obtaining an accurate picture of phenomena 
from mere verbal descriptions. These statements suggest the 
danger involved in having others do a considerable part of one’s 
research. It is true, of course, that research involves much 
routine work, some of which can be delegated, but if the data 
are collected entirely by others, the investigator loses contact 
with the object of study itself. This may mean that many im- 
portant relationships pass unobserved or that he lacks the in- 
timate knowledge of the processes involved, information 
essential to the formation of correct generalizations. The im- 
portance of this point in the school system of to-day with its 
exaggerated emphasis on administrative procedure can hardly 
be overemphasized. 

Importance of examining with care the common-place 
features of objects. The research worker should be vividly 
aware of the fact that he has been reared in an environment 
which predisposed him to look at phenomena in particular 
ways. The aspects of phenomena which one has been trained to 
observe may not be the important ones for the purposes of the 
investigation in progress, or the generalizations formed about 
them may or may not be true. So thoroughly habituated are 
persons to their ordinary ways of thinking about things, that 
it is exceedingly difficult to see conditions in any other than 
the customary light. Because of this very important psycho- 
logical fact, it is essential that those engaged in research be 
alert, not merely to the new and incidental features of objects, 
but to their more obvious features. 

In an analysis of the characteristics of good and poor 
teachers? made by one of the authors the investigator studied 


2A. S. Barr, Characteristic Differences in the Teaching Performance of Good and Poor 
ee of the Social Studies. Bloomington, Ill.: Public School Publishing Co., 1929. 
Dp. 127. 
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not merely the more conventional aspects of teaching—the as- 
signment, motivation, and provisions for individual differences 
—but also many of the less conventional phases of instruction: 
the number of times the teacher gestured, whether she stood or 
sat as she taught, the character of her comment, etc. He real- 
ized that, although the more conventional aspects of teaching 
are probably important, there are doubtless other less conven- 
tional phases of instruction of equal significance which have 
not been generally observed. Some persons were very much 
disturbed by the unconventionality of this study. The author, 
however, really desired to know why some teachers succeed and 
others fail, and did not wish to be hampered by the ordinary 
conventionalities of educational or investigational procedure. 

In his investigation of the Qualities Related to Success in 
Teaching, Knight ° studied a large number of the probable de- 
terminers of teaching ability, including the teacher’s age, ex- 
perience, salary, intelligence, professional information, profes- 
sional study, penmanship, normal-school marks, and interest in 
the subject taught, with the hope that he might find qualities 
related to teaching success. In choosing qualities for analysis he 
attempted to free himself from current conceptions of which 
factors were and were not related to teaching success. 

What the writers are attempting to emphasize here is the 
importance of freeing one’s self from the bonds of conventional 
thinking and procedure, and of looking with the greatest care 
at all of the ordinary and extraordinary concomitants of 
phenomena. 

Importance of seeing beyond the superficial aspects of ob- 
jects. Reference has already been made to the fact that in 
physics, objects with very different exteriors may operate ac- 
cording to the same principle. This observation appears par- 
ticularly important in the study of teaching, where the real 
determiners of effective learning and teaching seem not to be 
the immediately observable activities of the teachers and the 


3 F. B. Knight, Qualities Related to Success in Teaching. Teachers College Contribu- 
tions to Education, No. 120. New York; Bureau of Publications, Teachers College, 
Columbia University, 1922. Pp. 67. 
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pupils, but the less easily discernible uniformities which lie 
back of these activities. Many of the complications in the study 
of learning and teaching arise from the fact that the visible 
manifestations of these phenomena are so much in evidence 
that it is difficult to center attention upon the fundamental 
conditioning or underlying factors. In the preceding section of 
this chapter the desirability of noting carefully the more ob- 
vious features of things was emphasized, since one’s training 
and one’s conventional points of view may lead to the neglect 
of important aspects of phenomena. In the present section the 
importance of extending the search for vital relationships be- 
yond the superficial aspects of phenomena to underlying prin- 
ciples, conditions, and uniformities is emphasized. Objects 
which appear to be quite unlike may possess many similarities. 
The investigator must keep himself alert to the similarities of 
seemingly unlike situations. 

The importance of noting the uniform connection of 
events. All suspected relationships of events are suggested as 
the result of observing one thing to happen along with or 
shortly after or prior to another. As one changes the type of 
classroom discussion from that of general pupil-participation 
to uninterrupted pupil-reports, a decrease in the percentage of 
pupil-attention is observed, with the conclusion that the un- 
interrupted pupil-report is in some manner associated with 
pupil-attention. It is also noted that if pupils are required to 
repeat a great number of times certain types of activities, they 
eventually learn and if not they do not learn, with the con- 
clusion that drill is essential to certain types of learning. One 
notes when acquiring certain facts in history, geography, and 
grammar that they are soon forgotten, but that when one learns 
to skate, swim, or perform some other difficult act of skill, such 
activities are almost never forgotten, and one concludes that 
things learned functionally are retained longer than those ac- 
quired in the conventional, academic fashion, etc. The almost 
universal condition for the formulation of generalizations is 
that one observe the concomitant occurrence of conditions and 
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events. Such observations may be utilized in the formation of 
new hypotheses by enabling us to be more alert to the sequence 
of events. 

Moreover, although the careful noting of the concomitant 
occurrence of conditions and events is an important means of 
discovering hypotheses, one may be misled by attendant cir- 
cumstances. There is, for example, an interesting argument in 
progress at present in the field of learning about the function 
of repetition in learning. Early psychologists emphasized the 
fact that repetition was an essential condition for learning. 
Some present-day psychologists hold the view, however, that 
repetition (which we ordinarily associate with learning) is 
merely an incidental feature of learning and that it is only 
necessary because as repeated attempts to learn are made, the 
appropriate condition sooner or later is provided by random 
efforts, and learning takes place. Whether this contention is 
valid or not is aside from the present discussion. It is true, how- 
ever, in noting the sequence of events as a means of discovering 
new hypotheses, that the superficial features of phenomena are 
frequently mistaken for true causes when they are merely at- 
tendant circumstances or incidental to the real determiners of 
the results observed. The observer cannot be too careful in this 
respect. 

The importance of analogy in the formation of hypotheses. 
In reasoning from analogy one employs what is known about 
one object or relationship to explain what is not known about 
another. If two things resemble each other in one or more 
respects and a certain generalization is known to be true for 
one situation, it is considered to be true for the other. 
Reasoning from analogy is generally condemned as a method 
of proof; it is, however, a fruitful source of hypotheses. Refer- 
ence has already been made to the qualitative and quantitative 
differences in teaching performance. Although such a conception 
of teaching may or may not prove helpful, it arises out of a 
bit of analogical thinking in which the study of teaching is 
considered similar to the chemical analysis of physical objects. 
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Students of chemistry have been successful in studying physical 
objects by centering their efforts upon three fundamental as- 
pects of these objects: (1) the presence and absence of certain 
known qualities or constituents, (2) the quantitative variation 
of the constituents in objects, and (3) the arrangement of con- 
stituents, This suggests that teaching might be approached in a 
similar manner. The present emphasis upon diagnosis in edu- 
cation arose from its successful application in medicine and has 
proved a helpful concept. The theories of instincts, recapitu- 
lation, and gregariousness found in earlier educational discus- 
sions arose from the analogies derived from the study of 
biology. This list of illustrations might be greatly extended. 
The well-informed general reader of scientific literature will 
indeed find many helpful suggestions and analogies in the phys- 
ical and biological sciences. 

The fact that reasoning from analogy must be employed with 
great care has already received emphasis. The fallacy of ana- 
logical reasoning involves three rather common errors. In the 
first place, since analogy arises from noting the similarities of 
objects, it frequently may neglect important differences. In 
the second place, points of resemblance must be weighed and 
not merely counted. The mere number of resemblances between 
objects may or may not be important. Finally, the resemblance, 
between objects may be of a superficial or non-essential nature. 
Science is concerned with essential likenesses and differences. 

The importance of thinking about things. In the preceding 
sections of this chapter the authors have attempted to offer 
specific suggestions to aid the student in securing worth-while 
ideas about the nature and relationships of the objects of edu- 
cational research. One way is to be well-read, and another is 
to come into first-hand contact with phenomena by syste- 
matically observing them or experimenting with them. A third 
method is to examine carefully the common-place features of 
objects—the things that others may have passed by as un- 
important. One may also discover the antecedents of objects in 
some uniformity not immediately observed but brought to light 
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by probing beneath the surface of things; or one may get an 
idea in terms of an analogy taken from some related science. 
It is not easy to give helpful suggestions concerning these less 
well-analyzed aspects of scientific thinking, but it is hoped that 
the recommendations made may prove of some value to the 
reader. 

A less helpful and less concrete suggestion, but at the same 
time a fundamental condition for securing ideas, is to think 
about them. In the study of complex phenomena one may sè- 
cure the necessary insight into their essential qualities only 
after years of careful observation and painstaking investigation, 
during which time many hypotheses may have been tried and 
rejected. Kepler is said to have tried no less than nineteen dif- 
ferent hypotheses before he hit upon his theory of planetary 
motion. Even when one has worked with an object for a long 
time, he is frequently surprised to see clearly certain hitherto 
unobserved aspects. Almost every one has had the experience 
of seeing the vague impressions of his first contact with things 
become clear and precise upon further contact with the 
situations in question. The impressions become distinct only 
after one has been forced to consider them from many points 
of view. As a rule, it is only by prolonged efforts at thinking 
that one achieves any very worth-while ideas. 

Characteristics of a good hypothesis. To serve its intended 
purposes, the hypothesis must possess four important char- 
acteristics: (1) a good hypothesis is in agreement with the 
observed facts; (2) a good hypothesis does not conflict with 
any law of nature which is known to be true; (3) a good hy- 
pothesis is stated in the simplest possible terms; and (4) a 
good hypothesis permits of the application of deductive reason- 
ing. : 
(x) A good hypothesis is in agreement with the observed 
facts. Although not all of the facts have been observed in the 
formation of a hypothesis, in so far as they have been observed 
the hypothesis must be in agreement with the facts. A single 
unexplained conflict between fact and hypothesis is disastrous 
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to the latter. This principle is stated sometimes as requiring 
that the hypothesis be plausible and conceivable, and phrased 
thus is, of course, entirely satisfactory as long as one remem- 
bers that, although a hypothesis may appear inconceivable and 
absurd to the ordinary mind, it may agree with the observed 
facts and therefore be plausible to the trained worker. One of 
the most successful theories of physical science, for example, 
involves the most absurd suppositions. Gravity is a force which 
appears indifferent to intervening objects, and its action seems 
instantaneous. Notwithstanding these apparent absurdities, the 
law of gravitation seems well founded. In the formulation of a 
hypothesis, therefore, one is merely attempting to determine 
some simple colligation of the facts which is in agreement with 
the observed facts. 

(2) A good hypothesis does not conflict with any law of 
nature which is known to be true. The idea involved in this 
principle of hypothetical thinking is not very different from 
that expressed in the preceding principle, which emphasized 
that the hypothesis must not conflict with known facts. In the 
second characteristic of a good hypothesis, emphasis is placed 
upon the fact that the hypothesis must not conflict with the 
known laws of nature. It might be stressed here that there are 
many illustrations of apparently conflicting theories of learning 
and teaching in the field of education, ranging all the way from 
general theories of learning to details of the teaching process, 
from Gestalt psychology, behaviorism, and associationalism to 
questions of whether one should add up or down in learning to 
add, whether one should learn to speli by rule or rote memory, 
whether one learns best from the visual or auditory presenta- 
tion of materials, etc. These conflicting theories of learning and 
teaching arise partly out of the failure of research workers to 
produce hypotheses that are in agreement with fact, to place 
sufficient restrictions upon their generalizations (generalizations 
true for one set of conditions may not be true for another set, 
etc.), and to explain apparent conflicts. Not until it is proven 
that a generalization does not conflict with other commonly 
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accepted generalizations should it be accepted as true. In the 
application of this latter principle, one must not, of course, be 
misled by superficial appearances, since a slight modification 
of the hypothesis or a reformulation of the law with which it 
appears to conflict may bring about agreement. One may be 
only a special application of the other, or both may be merely 
special applications of some other unconceived and more funda- 
mental generalization. 

(3) A good hypothesis is stated in the simplest possible 
terms. This principle is suggested by the law of parsimony, 
which implies that one be stingy in the formulation of hy- 
potheses. It is also sometimes called Occam’s razor, after Wil- 
liam of Occam, a scholastic philosopher who said that entities 
are not to be multiplied beyond necessity. It is also called the 
principle of economy, which states that the ultimate general- 
izations of a science should be reduced to as small a number 
as possible. If the law of parsimony is not to be violated, the 
number of hypotheses must not be needlessly multiplied, thus 
the necessity for formulating each in the simplest and most 
fundamental terms. 

(4) A good hiypothesis permits the application of deductive 
reasoning. This principle is of the utmost importance because a 
hypothesis cannot be verified until deductions can be made 
from it which are capable of empirical verifications. Since the 
validity of a hypothesis is to be proved by its conformity with 
fact, it is entirely valueless unless stated in such form that its 
inferred consequences may be compared with the facts of ob- 
servation. There are two important ideas involved in this prin- 
ciple: one that a good hypothesis must lend itself to deductive 
reasoning, and the other that the hypothesis must be verifiable. 
Reference has already been made to the first of these ideas. 
Because of the importance of being able to verify hypotheses, 
much emphasis is also placed in logic upon the second idea. One 
can, for example, imagine all sorts of mystical forces operating 
in things, but inasmuch as one could never decide whether such 
forces exist or not, it would be a waste of time to indulge in 
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this type of imagination. To serve the purposes of science, 
therefore, a hypothesis must be capable of verification, as by 
comparing its deducible consequences with the facts of ob- 
servation. 

Testing of hypotheses. There are two fairly important 
means of testing hypotheses: (1) the study of hypotheses for 
logical consistency ; and (2) the study of hypotheses for agree- 
ment with fact. Although it probably goes without saying that 
the second of these means appears to be the more fundamental 
of the two, the first is not without value. 

Study of hypotheses for logical consistency. The study of 
hypotheses for logical consistency is a phase of thinking in 
which the average educational worker appears none too well 
trained. In the first place, this procedure consists of checking 
the logical character of the reasoning by which the conse- 
quences of hypotheses are deduced for verification. Although 
one will not usually desire to recast the argument in the form 
of a syllogism or proposition, a knowledge of the pitfalls and 
fallacies of deductive reasoning, which can be found in an 
elementary treatise on the subject of logic, will prove exceed- 
ingly valuable. This point is particularly important, inasmuch 
as the appeal to the facts of observation does not, of course, 
verify hypotheses, but merely the results deduced from them. 
After one has painstakingly compared the deduced conse- 
quences of an hypothesis with the observed facts, one must still, 
by careful reasoning, determine the status of the hypothesis. 
In the second place, the study of hypotheses for logical con- 
sistency involves checking them for agreement with the already 
known laws of nature. One of the characteristics of a good 
hypothesis, to which reference has already been made, is agree- 
ment with the established laws of nature. It must not conflict 
with the highest and simplest laws of good thinking, and it 
must not disagree with those principles of science which are 
considered valid beyond reasonable doubt. The fact that con- 
flicts exist means that the consequences of the theory are not 
verified if previous discoveries are correct, and one must, there- 
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fore, show that earlier discoveries are invalid before one can 
verify the hypothesis. Therefore, by studying the logical char- 
acter of the reasoning by which the consequences of hypotheses 
are deduced for verification, and the agreement with already 
known laws of nature, suggested inferences are tested in 
thought, for logical coherence, before they are tested in action. 

Study of hypotheses for agreement with fact. In the study 
of hypotheses for agreement with fact, one reasons that if the 
hypothesis is true certain facts, conditions, or relationships will 
be found; then one looks to see if these conditions are present. 
If it is found that the results agree with the theoretically de- 
duced consequences, and if there is no reason to believe that 
only the localized conditions in question could produce the 
results observed, then the probabilities that the hypothesis is a 
true generalization are relatively great. One may test the hy- 
pothesis by applying it to already known facts or by a new 
appeal to experience. The failure of educational theorists to 
keep this fact in mind has led to needless confusion over the 
relative merits of science and of philosophy in the study of 
educational problems. Science and philosophy employ the same 
mental techniques. Philosophy starts with the already available 
data of experience, formulates hypotheses in the light of these 
data, and returns to the data for verification. Science does like- 
wise, but if the data are incomplete, contradictory, or gathered 
under conditions that may cast doubt upon their trustworthi- 
ness, science makes a new appeal to experience, wherein the 
data are collected, recorded, and manipulated according to the 
conventions of science. The element of newness contributed by 
science is this new appeal to experience wherein new data are 
collected presumably with greater care than the data already 
available. 

The reader should also realize that although it is customary 
to assign some fixed position to the formulation of hypotheses 
in the sequence of mental steps which constitute scientific 
thinking, it is impossible to do so except in a very tentative 
way. When one is confronted with a problem, he begins to think 
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about it in terms of known facts. A hypothesis is formed, aud 
employed in the collection of additional data. These data may 
be found: (1) in one’s own experiences, recorded or unrecorded ; 
(2) in the recorded past experiences of other persons with 
similar problems based either upon incidental or systematic 
observation; or (3) in an appeal to new experience wherein the 
data are collected according to the conventions of science. New 
hypotheses may arise at any stage in the thinking process. In 
a single investigation many hypotheses may be formed, em- 
ployed for the time being, and discarded or modified in favor 
of new hypotheses. If the data already available are adequate 
for the testing of the hypothesis, no new appeal to experience 
will be necessary. Whether additional information is necessary 
will depend, of course, upon the nature of the problem and the 
character and completeness of the available data. 

Concluding statement, In this chapter an attempt has been 
made to indicate briefly the nature and importance of the use 
of hypotheses in educational research. A hypothesis is a guess, 
supposition, or tentative inference as to the existence of some 
fact, condition, or relationship, which serves to explain the facts 
known to exist about some object under investigation. The 
hypothesis serves the important function of limiting the field 
of investigation, of sensitizing the observer to pertinent facts, 
and of offering a simple means of linking the facts together. 
The importance of being well informed, of examining with great 
care the common-place features of objects, of seeing beyond 
the superficial aspects of objects, of noting the uniform con- 
nection of events, of analogical reasoning, and of thinking about 
things was eniphasized as a means of getting ideas about the 
antecedents of the objects of educational research. The char- 
acteristics of a good hypothesis are: (1) it is in agreement with 
the observed facts; (2) it does not conflict with any law of 
‘nature which is known to be true; (3) it is stated in the 
simplest possible terms; and (4) it admits of deductive reason- 
ing. There are two fairly important means employed in the 
testing of hypotheses: (1) the analysis of hypotheses for logical 
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consistency, and (2) the study of hypotheses for agreement 
with fact. Much of the theorizing about the relative merits of 
science and of philosophy in educational research has arisen 
out of a misconception of the nature and use of the hypothesis 
in thinking. 


PROBLEMS AND EXERCISES 


1. Formulate three hypotheses, which merit testing, relating to im- 
portant educational issues or problems of to-day. 

Select one of the investigations cited in the bibliography at the 
end of this- chapter, or some other equally satisfactory study, and 
evaluate the hypotheses which appear there, in terms of the prin- 
ciples developed in this chapter. 
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CHAPTER V 


THE CLASSIFICATION OF RESEARCH METHODS 


Introduction. After the investigator has selected a problem 
for study and has become thoroughly familiar with its possi- 
bilities through a careful bibliographical survey of the field 
represented, it is necessary to determine what methods or 
techniques of collecting data are most appropriate in solving 
the problem or in verifying the hypotheses formulated. The 
present chapter will consider the classification of research . 
methods; illustrative enumerations of research procedures 
in the fields of education, psychology, and sociology ; and 
some information concerning frequency of the use of investiga- 
tional techniques. Finally, the authors’ conception of a pattern 
of research methods, as determining the organization of sub- 
sequent chapters of this book, will be outlined. 

At this time no definition or explanation of the various types 
of research is attempted, since this is the function of the 
later chapters. In the interests of simplicity and variety of 
expression, no distinction is made between such terms as 
methods, techniques, procedures, and types of research, al- 
though the term devices is used frequently to indicate less 
fundamental or less basal procedures such as data-collecting 
instruments in the form of questionnaires, check-lists, score 
cards, etc. These statements raise a question concerning the 
meaning of the term data, as it is used in educational investi- 
gation. Monroe and Engelhart have provided good illustrations 
of various types of data.* 

Data include all concepts, facts, and principles used in thinking out 
the answers to thought questions. In other words, data are the things 


1 Walter S. Monroe and Max D. Engelhart, The Techniques of Educational Research 
p. 27. University of Illinois Bulletin, Vol. XXV, No. 19. Bureau of Educational Researc! 
Bulletin, No. 38. Urbana, Ill.: University of Illinois, 1928. 
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we use in thinking. The meaning of these statements will be more 
apparent if we note some of the types of data. 

1, Ages of school children 

2. Scores made on standardized tests 

3. School marks, and other facts from school records 

4. Answers to questionnaires 

5. Expenditures for educational purposes 

6. Assessed values and tax rates 

7. Measures of school buildings 

8. Statements of beliefs or opinions 

g. School laws 

to, Counts of things such as school children, number of pages, etc. 
rz. Principles 

12. Descriptions of schools, events, etc. 

13. Observations 

14. Historical information 

15. Errors in children’s compositions 

16. Stenographic reports of lessens 


The same authors emphasize that when the problem has been 
adequately defined, at least the types of data needed for its 
solution are indicated. Therefore “collecting data” does not 
mean bringing together whatever data may be easily accessible, 
but rather, those specified by the problem. The illustrations of 
data in the preceding list indicate that a variety of techniques 
are necessary in bringing together needed information. More- 
over, the solution of a given problem may involve the collec- 
tion of several types of data and the use of an equal number 
of research techniques. 

Classification of educational research. A decade ago there 
was a limited amount of material in print for the investigator 
interested in the methodology of educational research. Now 
the number of discussions of investigational techniques,? many 
with an individual classification of types of research, is so large 
and varied in character that the reader may feel considerable 
confusion about the meaning, terminology, and classification 
of research in education. This confusion appears to develop 

2 For an annual bibliography on this subject, see the September or October number of 
the Journal of Educational Research, for example, Carter V. Good, ‘Selected Bibliography 


on the Methodology of Educational Research and Related Problems, 1934-1935,” Journal 
of Educational Reseerch, XXIX (October, 1935), 156-70, 
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from three conditions. (1) There may be disagreement as to 
the scope of such expressions as research, scientific, experi- 
mental, etc. One frequently reads, for example, that there are 
four types of research: (a) philosophical, (b) scientific, (c) 
historical, and (d) statistical. Are the historical, philosophical, 
and statistical methods of research scientific? Is there an his- 
torical, a philosophical, a statistical method of research? Ir 
statistical procedure a method of research or a method of 
analyzing data applicable to several types of research? (2) 
The use of the name of a phase or part of the research process 
to describe the total activity—descriptive, causal, analytical, 
questionnaire, etc —is often misleading. (3) The material may 
fail to indicate clearly the point of view from which classifica- 
tions are made. 

Methods of research may be classified from many points of 
view: the fields to which applied: education, history, philos- 
ophy, psychology, biology, etc.; purpose: description, predic- 
tion, determination of causes, determination of status, etc.; 
place where it is conducted: in the field or in the laboratory ; 
application: pure research or applied research; data-gathering 
devices employed: tests, rating scales, questionnaires, etc. ; 
character of the data collected: objective, subjective, quantita- 
tive, qualitative, etc.; symbols employed in recording, de- 
scribing, or treating results: mathematical symbols or language 
symbols; forms of thinking: deductive, inductive, etc.; control 
of factors : controlled and uncontrolled experimentation; meth- 
ods employed in establishing causal relationship: agreement, 
difference, residues, and concomitant variation. Where there is 
a shifting of the point of view in a given classification, without 
warning or explanation, the result almost always is confusing ; 
it is similar to that of a shelf of books classified in several 
ways simultaneously: color, size, cost, title, etc. Although the 
problem here described is primarily one of terminology, it is 
nevertheless exceedingly important, especially when one wishes 
to describe the procedures used in one’s own investigations or 
to give systematic instruction in the methodology of research, 
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The preceding paragraph sets forth the point of view ex- 
pressed in the introduction to a symposium on the classification 
of educational research in which eighteen individuals partici- 
pated.* It seems significant for comparative purposes to 
enumerate the classifications prepared by a number of the 
contributors to this symposium, including Almack, Ayer, 
Dashiell, Gates, Johnson, Kelley, McCall, Ruch, Symonds, 
Trabue, Whitney, Woody, Kilpatrick, and Freeman. 

John C. Almack lists as techniques of research: historical 
(applied to documents), normative (applied to data of record), 
experimental (applied to experiment), and conceptual or classi- 
ficatory (applied to natural experiment). He lists statistics, 
record forms (questionnaire), calculating machines, charts 
(correlation), and instruments of precision as devices or tools 
of research.* 

Fred C. Ayer considers five phases of research activity im- 
portant in school administration: analysis, refinement of 
instruments, experimentation, practical application, and dis- 
semination. 

J. F. Dashiell discusses direct observational (other than ex- 
perimental), experimental, survey, historical, and speculative. 
methods of research. 

Arthur I. Gates states that representative tests or devices 
used by scientists, philosophers, historians, statisticians, and 
others engaged in the pursuit of truth include methods of de- 
termining validity of facts previously reported, methods of 
securing valid data by means of observation of a present event, 
and methods of securing valid conclusions or generalizations 
or principles. 

Palmer O. Johnson has prepared an unusually complete out- 
line of procedures in educational research: 


A. Documentary analysis 
1. Historical 
3 A. S. Barr and Others, “A Symposium on the Classification of Educational Research,” 


Journal of Educational Research, XXIII (May, 1931), 353-82; XXIV (June, 1931), 1-22. 
4 Also see John C. Almack, Research and Thesis Writing. Boston: Houghton Mifflin 


Co., 1980. Pp. xviii + 310. 
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2. Legal 
(a) Statutes 
(b) Decisions 
3. Courses of study 
4. Records and reports 
(a) Institutional 
(b) State 
(c) Federal 
(d) Private foundations 
(e) Surveys 
(f) Educational associations 
(g) Foreign school systems 
5. Textbooks 
B. Indirect observation 
1. Check-lists 
2, Correspondence 
3. Questionnaires 
4. Rating scales 
(a) Graphic 
(b) Order of merit ranking 
(c) Man-to-man comparison 
(d) Paired comparisons 
(e) Range of values 
C. Direct observation without instrumental control 
1. Ordinary observation 
2. Unstandardized interview 
D. Direct observation with instrumental control 
1. Experimental methods 
(a) One-group 
(b) Equivalent-groups 
(c) Rotation 
Learning 
Physiological 
Psycho-physical 
Standardized interview 
Test methods 
Verbal reports 
Seapets methods 
Case method 
Case-group method 
Activity analysis 
Job analysis 
Field studies 
Survey 
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‘Truman L. Kelley distinguishes between four types of re- 
search: the field of logic, metaphysics, pure mathematics, etc. ; 
history; experimental science; and forecasting.’ 

William A. McCall’s classification of research according to 
the methods or instruments employed is as follows: € 


One-group experimental method 
Equivalent-groups experimental method 
Rotation-groups experimental method 
Varying-concomitants method 
Correlation and partial-correlation method 
Causal-investigation method 

Case method 

Genetic method 

Bibliographical method 

Comparative method 

. Analytical method 

Statistical method 

Questionnaire method 

Test method 

Rating method 

Quantitative method 

Historical methods 

Sociological methods 

Psycho-physical methods 

Scientific methods 

. Philosophical methods 
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G. M. Ruch names historical, philosophical, and scientific 
(experimental) research as fundamental to the solution of 
educational problems. He considers statistical method a power- 
ful tool in both description and inference and the ally of all 
methods. 

Percival M. Symonds lists historical investigation, descrip- 
tive survey, analytical survey, discovering relationships, ex- 
perimentation, and development of techniques as methods of 
research.” 


5 Also see Truman L. Kelley, Scientific Method. New York: The Macmillan Co., 1932. 
Pp. x + 234. 

6 Also see William A. McCall, How to Measure in Education and How to Experiment 
in Education. New York: The Macmillan Co., 1922, 1923. Pp. xiv + 416, xvi + 282. 

7 Also see Percival M. Symonds, “A Course in the Technique of Educational Research,” 
Teachers Coilege Record, XXIX (October, 1927), 24-30. 
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M. R. Trabue mentions as special tools and methods used 
in the selection and collection of data: experimentation, 
measurement, observation, interview, questionnaire, corre- 
spondence, recording, and reading. He includes in the process 
of analysis, evaluation, and organization of data the statistical 
and individual (case) methods. 

F. L. Whitney’s classification includes historical, descriptive, 
experimental, philosophical, and predictive research.* 

Clifford Woody’s outline follows: 


I. According to techniques employed in the collection of the data 
(Survey and case studies may be classed in either category, de- 
pending upon the techniques used.) 

Iı. Questionnaire studies 
(a) Letters 
(b) Inquiry or information blanks 
(c) Interviews 
2, Observational studies 
(a) Activity analysis 
(b) Job analysis 
(c) Analysis of recorded observed responses 
3. Experimental studies 
(a) Uncontrolled experimental 
(b) Controlled experimental 
II. According to the method of treatment of data 
1, Historic 
2. Philosophic 
3. Statistical 


As one of the critics of the symposium to which the preceding 
writers contributed, William H. Kilpatrick distinguishes be- 
tween three kinds of research: scientific, historic, and philo- 
sophic. 

Another critic of the symposium, Frank N. Freeman, out- 
lines a classification of educational research based on method: 


1. Description 
(a) Historical techniques: discovery and criticism of documents 
(b) Survey of contemporary conditions—techniques: examination 
8 Also see F. L. Whitney, Methods in Educational Research. New York: D. Appleton 


and Co., 1931r. Pp. xviii + 336. Chapter III includes a tabular classification of twenty-two 
concepts of methods of educational research taken from twelve sources. 
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of records, observation, testing and measurement, statistical 
and graphic formulation 
(c) Account of genesis or development—techniques: observation, 
keeping and examination of records, testing and measurement, 
statistical and graphic formulation 
2. Analysis and comparison 
(a) Analysis of particular processes—techniques: observation and 
laboratory experiment 
(b) Individual and group comparison—techniques: observation, 
testing or rating, statistical and graphic formulation of results 
(c) Case study—techniques: observation, keeping and examination 
of records, testing and synthesis of results 
3. Systematic account and explanation of relations 
(a) Statistical analysis of relations—techniques: techniques of 
gathering data in common with other methods, statistical and 
graphic methods of showing correlation 
(b) Experimental variation—technique: comparison of individuals 
or groups under controlled conditions, by the introduction of a 
single variable, the use of a parallel control group or the rota- 
tion of groups 
4. Forecasting (?) 
Doubt has already been expressed as to the acceptability of fore- 
casting as a type of research. 
5. Evaluation (?) 
This is what commonly goes by the name of philosophy. It is a 
question whether evaluation should be separated from the foregoing 
types of research. It should frequently stand as the culminating 
stage of particular investigations. Evaluation as often carried on 
is speculation or the expression of prejudice rather than research. 
It is possible, however, that one procedure may be set apart as 
different from the preceding and may be made the basis of fur- 
ther evaluation. This procedure is the synthesis of the results 
of a number of researches so as to apply to practical problems the 
facts which bear on them from many angles, 


For the sake of completeness a few classifications not rep- 
resented in the symposium may be mentioned briefly. Space 
does not permit individual comment on the adequacy of the 
classifications represented in the symposium or on the outlines 
of research methods which follow. A general statement is made 
later in this chapter concerning the inconsistencies present in 
similar classifications. 


Monroe and Engelhart give illustrations of subjective, ana- 
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lytical, experimental, historical, interview, legal, questionnaire, 


survey, test-construction, and observation techniques.° 
A. Subjective data 
1. Formulation of criteria to be used as a basis for estimates 
2. Use of criteria 
B. Objective data 
1. Techniques employed in collecting 
(a) Analysis 
(1) Analysis of textbooks 
(2) Analysis of pupil performances 
(3) Analysis of records 
(b) Experimental procedures 
(1) One-group method 
(2) Equivalent-group method 
(3) Rotation method 
(c) Historical 
(d) Interview 
(e) Legal 
(1) Statutes 
(2) Decisions 
(f) Questionnaire 
(1) Opinion 
(2) Facts 
(g) Survey 
(h) Test construction 
(1) Scale 
(2) Tests 
(i) Observation 
2. Forms in which used 
(a) Raw 
(b) Transmuted 


Crawford’s classification includes experimental, historical, 
psychological, case study, survey, curriculum making, job an- 
alysis, interview, questionnaire, observation, measurement, 
statistical, tabular and graphic, and library techniques. 

Waples and Tyler discuss techniques of analysis, reading 
and recording, observation, personal interview and group con- 


9W. S. Monroe and M. D. Engelhart, The Techniques of Educational Research, pp. 
28-38, University of Illinois Bulletin, Vol. XXV, No. 19. Bureau of Educational Research 
Bulletin, No. 38. Urbana, Ill.: University of Illniois, 1928. 

10C. C. Crawford, The Technique of Research in Education. Los Angeles, Calif.: 
University of Southern California, 1928. Pp. 320. 
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ference, obtaining written statements by question blanks, check- 
list, sampling, classification, summarizing, evaluation, indi- 
vidual judgment, group rating, comparison, space and fre- 
quency counts, testing, and experimentation. 

Koos recognizes the questionnaire, experiment, measurement, 
documentary analysis, case, mathematics, and survey. 

Abelson’s rather novel scheme includes descriptive, causal, 
evaluative, constructive, and integrative methods.'* 

Schluter’s classification includes experimental, survey, his- 
torical, and philosophical methods of research. 

The National Committee on Research in Secondary Educa- 
tion in 1927 discussed the following types of research prob- 
lems and methods: historical, experimental, philosophical, 
survey, case, and questionnaire.’ 

A psychologist has analyzed the methods of child study as 
follows ; +° 


Incidental observation 

Biography 

Systematic observation 

Questionnaire 

Psychoanalysis 

Case history 

Direct measurement and simple tests 
Tests of complex functions 

Ratings 

to. Experiment 

11. Experiment involving random control groups 
12. Experiment involving paired control groups 
13. Control by statistical devices 

14. Factor analysis 


OMI AMAw Dy 


11 Douglas Waples and R. W. Tyler, Research Methods and Teachers’ Problems, 
Chapter VI. New York: The Macmillan Co., 1930. 

12L. V. Koos, The Questionnaire in Education, New York: The Macmillan Co., 
1928, Pp, viii + 178. 

183 H. H. Abelson, The Art of Educational Research, Chapter III. Yonkers-on-Hudson, 
New York: World Book Co., 1933. 

14W. C. Schluter, How to Do Research Work. New York: Prentice-Hall, 1927. 
Pp. vii + 137. 

15 A. J. Jones and Others, An Outline of Methods of Research with Suggestions for 
High School Principals and Teachers. Bureau of Education Bulletin, No. 24, 1926. 
Washington: Bureau of Education, 1927. Pp. vi + 31. 

16 J. E. Anderson, “The Methods of Child Psychology,” Handbook of Child Psychology, 
pp. 11-12. (Second revised edition.) Edited by Carl Murchison. Worcester, Mass.: Clark 
University Press, 1933. 
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Two other psychologists distinguish between eight methods 


of research in child psychology: observational, rating, experi- 
mental, test, questionnaire, clinical, introspective, and psycho- 
analytic." 


A sociologist has analyzed the methods of sociological re- 


search applicable to a study of child-development under the 
heading of methods of research, techniques for collecting data, 
and approaches: +8 


Dapena 


in 


Methods of Research 


The historical method 

The survey method 

The case method 

The statistical method 

The experimental method 
Combinations of the above 


AVEO 


Techniques of Collecting Data 


The questionnaire: mailed, as an interview outline 

The interview (questionnaire outline; case-history outline) 
Autobiographies 

Moving pictures 

Tests and rating scales 

Observation: participant observer, controlled experimentation, de- 
tached observation 

Documentary evidence: diaries, letters, census information, court 
proceedings, government publications (national, state, local), pri- 
vate agency reports, research studies, etc. 

Combinations of the above techniques 


Approaches 


Philosophical or theoretical 
Historical 

Anthropological 

Economic 

Biological 


17G. D. Stoddard and Beth L. Wellman, “Methods of Research in Child Psychology,” 
“Methods and Techniques of Educational Research,” Review of Educational Research, 


VeRO DH 


IV (February, 1934), 65-71. 


18R. G. Foster, “Objective Methods of Sociological Research Generally Applicable to 


Child-Development Studies,” Journal of Educational Sociology, IX (October, 1935), 79-87. 
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6. Psychological 
7. Psychiatric and psychoanalytic 
8. Sociological (contemporary culture) 


Another sociologist, Fry, includes in the techniques of social 
investigation direct observation, the interview, questionnaires 
and enumerations, experimentation and tests, and combined 
methods of study such as the social survey.?® 

Two excellent bibliographies on methods and techniques of 
educational research are available. One reviews the application 
of appropriate research techniques to investigations in the 
fields of curriculum building, teacher personnel, school organi- 
zation, special methods in the elementary school, secondary- 
school methods of teaching, school finance, building surveys, 
educational tests, child psychology, pupil personnel and guid- 
ance, school law, library work, and the laboratory. Of course, 
a number of individual classifications of research methods are 
apparent in these reviews.” The bibliography by Barr and 
Rudisill is grouped under a working classification with sub- 
headings similar in most respects to the scheme employed in 
the present chapter.?* 

Frequency of use of research enean Before outlining 
the classification of research techniques to be followed in this 
book, it is of interest to cite certain attempts to determine 
frequency of use of the various investigational methods. The 
reader should be cautioned that these analyses are partly sub- 
jective, since they are based upon the given author’s individual 
conception of educational research and of investigational 
methods. In all probability the resulting frequencies in the 
classification would be different if the analysis were repeated 
by another worker. Granted that there is no evidence concern- 
ing the reliability of these classifications, such frequency tabu- 


19C. L. Fry, The Technique of Social Investigation. New York: Harper and Bros., 
1934. Pp. xiv + 315. 

20 Methods and Techniques of Educational Research,” Review of Educational Re- 
search, IV (February, 1934), 1-119. 

21 A. S. Barr and Mabel Rudisill, An Annotated Bibliography on the Methodology of 
Scientific Research as Applied to Education, Bulletin of the Bureau of Educational 
Research, No. 13, June, 1931. Madison, Wis.: University of Wisconsin. Pp. 129. 
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lations possess at least historical interest and emphasize the 
desirability of developing and using a simple, consistent classi- 
fication of methods—of distinguishing between fundamental 
modes of attack on educational problems and mere devices 
used in the collection or analysis of data. Confusion of major 
methods and data-collecting devices has been frequent. 

Koos reports an analysis of 581 studies in terms of investi- 
gational methods employed.” His findings are reported in 
Table VII, which is self-explanatory. 


TABLE VIL 
NUMBERS or STUDIES UTILIZING THE DIFFERENT METHODS OF INVESTIGATION 
AS SOURCES AND THE NUMBERS AND PERCENTAGES OF STUDIES TREATED 
QUANTITATIVELY 


Total Number Per Cent 


Method of Investigation Number | Quantitative | Quantitative 
Questionnaire..........5.0eeeee eee 143 141 98.6 a 
Experiment... ...-.s0eesererererenee 137 134 97-8 
Measurement. . . 109 106 97:3 
Documentary analysis........-.++-++ 145 120 82.8 
Case method........--eeeeeeeeseeeee 13 7 53-8 
Othiersa: o 55 Vek tht als cp NANETA ATS, 34 34 100.0 

All Methods. .......--++eeeeeees 581 542 93-3 ‘ 


Bixler presents an analysis of the basal methods of research 
and of the techniques of collecting data found in one hundred 
Doctors’ dissertations accepted during 1924-25. The results 
are found in Table VIII. It is recognized that some theses 
employ more than one method and are therefore represented 
more than once in the tabulation. 

Douglass has made a valuable analysis, reported in non- 


22L. V. Koos, op. cit., p. 53- 
23H. H. Bixler, Check Lists for Educational Research, pp. 85-7- New York: Teachers 


Collee, Columbia University, 1928. 
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tabular form, of types and fields of secondary-curriculum re- 
search in 1929." 


TABLE VII 


ANALYSIS OF THE METHODS OF RESEARCH AND OF TECHNIQUES OF COLLECTING 
Data USED IN ONE HUNDRED DISSERTATIONS 


Teachers 
Eleven College 
Method or Technique Univer- | Columbia 


sities | University | Total 


N=27 N=73 |N=r00 


I. Methods of research 


A. Historical method. ...............5 6 15 21 
B. Experimental method 
1. One-group. 3 9 12 
2. Equivalent-groups. . 8 13 21 
3. Rotation......... o o o 
C. Case method....... 2 7 9 
D. Case-group method. . o I I 
E. Investigational methods. . 20 56 76 


II. Techniques of collecting data 
A. Measurement 
x. Evaluation and selection of exist- 
Rng teSt See cate E trea bate lets II 27 38 


2, Construction of new tests... 10 13 23 
3. Administration of tests... 12 25 37 
As SCODE OF CEBES: a miclsit sik ates 12 20 32 
B. Interviews 5 Ir 16 
C. Questionnaires. . II 3I 42 
1. Necessity for, and value of, shown 6 16 22 

2, Scientific selection and arrange- 
ment of items 3 3 6 
3. The original form presented. . . 5 15 20 
D. Judgments or ratings 9 14 23 
I. Qualifications of the judges 3 8 Ir 

E. Personal observation, including sten- 
ographic reports .................. 3 8 Ir 


Franke and Davis provide an interesting tabulation of 
magazine articles by decades from 1890 to 1939 in terms of 


24H. R. Douglass, “Types and Fields of Curriculum Research in Secondary Education 
During 1929,” School Review. XXXVIII (November, 1930), 656-62. 


CLASSIFICATION OF RESEARCH METHODS 219 


the methods used in educational research.** For present pur- 
poses, only the analysis for the decade 1920-29 is reproduced 
in Table IX. The authors call attention to: (1) the compara- 
tively small amount of research using the historical, legal, com- 
parative-survey, and case-study methods (this is true of all 
year-periods); (2) the relatively large number of articles 
which used the questionnaire, correspondence, and interview 
methods, during the first two periods, 1890-1899 and 1900- 
1909 (a marked decline of these methods is noted during the 
last two periods); (3) the marked increase of statistical 
method from 1910 to the present time; (4) the rapid rise of 
experimental techniques from 1910 to 1919, with some decrease 
from 1920 to 1929 (this decline during the last period is prob- 
ably due to the influence of the testing movement). 

Good’s tabulation by five-year periods of 397 Teachers Col- 
lege (Columbia) Doctors’ dissertations in terms of research 
techniques employed forms Table X.°° 

Tt will be noted that the relative frequency of use of re- 
search methods in Doctors’ dissertations at Teachers College, 
Columbia University, is not the same in every instance as that 
reported for magazine articles by Franke and Davis.?* In the 
series of studies analyzed in Table X, it is evident that the 
purely philosophical or subjective thesis is infrequent and is 
becoming increasingly so. The historical and legal methods are 
a fair proportion of the total, although they are not employed 
frequently in secondary education. Former interest in philo- 
sophical, theoretical, and historical problems has shifted to 
legal and foreign aspects of education, but the technique in- 
volved is much the same. The questionnaire-correspondence 
method has been used in one-half of the Doctors’ dissertations 
printed in this series since 1924. This seems an overworking 
of a useful device, especially in secondary education. Students 


25 Paul R. Franke and Robert A. Davis, “Changing Tendencies in Educational Re- 
search,” Journal of Educational Research, XXIII (February. 1931), 133-745- 

26 Carter V. Good, “Fields and Types of Research in Education, 1918-1931,” Journal 
of Educational Research, XXIV (June, 1931), 33-43- 

27 Paul R. Franke and Robert A. Davis, op. cit. 
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TABLE IX 


DISTRIBUTION OF THE AMOUNT OF RESEARCH PRODUCED IN 1920-1929 ANALYZED 
ACCORDING TO METHODS 


Method Articles Pages 
í 


Number | Per Cent | Number | Per Cent 


Historical—Legal........-.-.--+- 17 .85 154 80 
Comparative—Survey....-+-..++- 88 4.38 774 4.00 
Questionnaire—Correspondence— 

Interview........-- | 303 15.06 2988 15.86 
Statistical... 852 42.37 7798 40.34 
Experimental 514 25.56 4932 25.11 
Case Studies 31 1.54 382 1.98 
Job Analysis: 

Social Survey... iaria ane 206 10.24 2305 11.92 

TORE NER ve As 2011 100 19333 100 


— alle 


TABLE X 


CLASSIFICATION OF TECHNIQUES OF RESEARCH IN 397 TEACHERS COLLEGE 
(COLUMBIA) CONTRIBUTIONS To EDUCATION 


Period Total 
Technique times 
1905 | 1910 | x915 | 1920 | 1925 | t930 technique 

1909 | 1914 | 1919 | 1924 | 1929 as 
Philosophical........ 6 5 o 3 7 o 21 
Historical, legal...... 17 T: 8 13 44 2 101 
Questionnaire. MU a 8 6 5 Ir | 101 28 155 
Statistical. ... 10 20 25 49 | 106 41 341 
Experimental... ... I 3 4 14 36 8 62 
MEMORIA ra eaae o o o 4 3 2 9 
o ng 2 o I4 o 17 
T I 4 7 30 13 56 

23 | 31 | 29 | 57 |213 | 44 397* 


*This, the total number of theses, is less than the total number of times the various techniques 
were used. 


il ie 


i 
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seem to recognize the criticism aroused by indiscriminate use 
of the questionnaire and seek to avoid the term by substituting 
such labels as question forms, data sheets, research blanks, and 
history blanks. Naturally statistical procedures have been em- 
ployed in a majority of the theses. More experimental studies 
are needed, especially in secondary education. This is a type of 
research little used before 1920 in this series of dissertations, 
and apparently not at all in secondary education before 1920. 
There are real possibilities in wider use of the case method, 
which is recent in its application to education. Within the five 
years from 1925-1930 the technique of analysis (activities, text- 
books, difficulties, etc.) was applied more frequently than be- 
fore, In certain instances, however, this method proves to be 
a mechanical tabulation of frequency counts hardly in keeping 
with the sort of problem-solving contribution expected of the 
doctoral candidate. A number of graduate students partici- 
pated in school surveys between 1925 and 1929 and used por- 
tions of the data accumulated as Doctors’ dissertations. Valu- 
able field experience may be secured in this way. 

Certain comparisons of the data of Tables IX and X are in 
order, Evidently many more graduate students than writers of 
magazine articles use historical or legal techniques of research, 
There is no evidence during the decade 1920-1930 of a decrease 
in the use of the overworked questionnaire methods by gradu- 
ate students, in fact, quite the contrary. It is to be expected 
that theses will emphasize objective statistical treatment of 
data more than is the case in magazine articles, although such 
procedures are common in both types of educational literature. 
In both articles and theses experimental techniques have been 
on the increase. Case studies have been relatively infrequent in 
the two analyses reported above, which also agree approxi- 
mately on the extent to which the various techniques of an- 
alysis have been employed. 

Classification of research methods used in this book. Fore- 
going portions of this chapter have outlined the various ways 
in which contemporary workers in the field of educational re- 
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search have classified methods of investigation. It is recognized 
that some confusion may result for the beginner from such a 
diversity of classifications, but such categories are useful for 
comparative purposes and historical interest. The student 
should remember, however, that all fundamental research 
shares the goal of truth and the elements of critical reflective 
thinking. 

No particular profit would result from, and indeed space is 
not available for, an attempt to evaluate these enumerations of 
research procedures. The critical reader may examine any de- 
sired classification in terms of internal consistency or con- 
formity to the steps in problem solving, or compare it with 
any other plan or with that employed in this book. It becomes 
immediately apparent that many of the foregoing outlines fail 
to distinguish between basal or fundamental approaches to 
problem solving (as in the case of the historical and experi- 
mental methods), devices for collecting data (questionnaires, 
check-lists, rating scales, etc.), and procedures for analyzing 
and interpreting data (for example, statistics, logic, and phi- 
losophy). Certainly the classifications cited at the beginning of 
this chapter in terms of fields involved, place where conducted, 
application made, etc., have little value for the present pur- 
pose. 

_ No attempt is made to characterize one method as more 
_ important than another, since the purpose to be served and 
the conditions to be met must determine the value of a re- 
j search method in a given situation. The’ value of a machine 
gun, a rifle, a shotgun, and a revolver as a weapon must be 
determined in relative rather than absolute terms. The diffi- 
culty of establishing a sharp line of demarcation between fun- 
damental methods of research and minor techniques or devices 
for collecting or analyzing data will be recognized ; since, under 
the direction of an expert, what was a minor technique in a 
different situation or in the hands of a beginner—the case 
study, for example—may become a major procedure in the ap- 
proach to truth. Therefore, the present arrangement of in- 
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vestigational procedures is intended merely as a working 
classification of the various methods or techniques of collecting 
educational data, without any claim for finality. 

Rarely does a given study represent a single method of in- 
vestigation, Frequently several modes of attack must be em- 
ployed in the collection of data adequate for the solution of a 
problem. Symonds appropriately cautions the worker concern- 
ing the current tendency to overemphasize differences in re- 
search techniques: °° 


The customary classifications of research techniques into types of 
research overemphasize differences which are matters of relative em- 
phasis only. In certain cases the whole investigation comprises a re- 
sumé and evaluation of past events, and we have historical research. 
Even in an experimental inquiry, however, one should precede his 
work by knowing what others have done in the past on the same 
topic or in the same field. Again all research should include statistical 
methods, although at times this use of statistics may be exceedingly 
elementary and simple. In historical study, however, which would 
seem to be as far removed from the statistical as possible, one should 
not draw conclusions on the basis of a single event. For example, one 
should not’ draw conclusions as to the nature of the academy in the 
eighteenth century from the description of one such academy. The 
question of sampling arises wherever generalizations are to be made, 
and sampling is a statistical issue. 


In attacking such an educational problem as adaptation of 
the instructional program to varying needs, it is clear that four 
major approaches are possible: (1) to examine the evidence 
and experience of the past as an aid in analyzing and inter- 
preting the present situation (historical research); (2) to 
canvass present practice with respect to plans for dealing with 
such differences, or to set up norms or central tendencies 
through testing and measurement against which the extent of 
differences may be checked (normative-survey research) ; (3) 
under controlled conditions, with only a single variable, to 
try out in classroom or laboratory different ways of adjusting 
to varying pupil needs in order to determine the effectiveness 


28A. S. Barr and Others, “A Symposium on the Classification of Educational Re- 
search,” Journal of Educational Research, XXIII (May, 1931), 375- 
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of a given procedure (experimental research); (4) to deter- 
mine relationships through statistical manipulation of data, 
correlational calculations, for example, or to diagnose the origin 
or cause of differences through intensive case or genetic study 
or causal-comparative group analysis, and possibly to follow 
such analyses by remedial prescription and practice (types of 
research especially adapted to studying complex causal rela- 
tionships). ji 

Some readers may say that this is an academic and logical 
rather than a functional and psychological classification, but 
in reality this fourfold division is as functional as problem- 
solving itself, involving as it does the major or fundamental 
modes of attack on educational problems. It is as psychological 
as the sequence and pattern of reflective thought,2° since later 
paragraphs show the place of a variety of procedures in the 
collection, analysis, and interpretation of data. Into this rea- 
sonably consistent and complete pattern of scientific method 
may be blended with little difficulty each of the research classi- 
fications enumerated earlier in this chapter. In-so doing, logic 
and philosophy are not excluded from the picture, but have a 
definite place and contribution, as pointed out later. The para- 
graphs which follow outline briefly this simple classification 
of research methods. Later chapters include detailed discus- 
sions of these techniques. ts 

1. Historical research. The historical method is considered 
a major approach to educational truth. Much of the emphasis 
of recent years in historical research has been diverted to the 
study of legal aspects of education, including statutes and 
court decisions. Both the more general historical problems and 
specialized legal research involve painstaking examination of 
the original sources and documentary study. Analytical works 
of a bibliographical and summary nature may well be classified 


29 J. Stanley Gray, “A Neglected Phase of Educational Research,” Journal of Educa- 
tional Research, XXIX (October, 1935), 83-90. Points out that the steps in problem 
solving or good thinking are essentially the same, and urges that each investigational step 
or procedure be recognized and evaluated in the light of its significance and relationship 
to the complete act of problem solving. 
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under this heading. Critical summaries require research ability 
of high order, if the work of analysis and synthesis is well 
done, and are valuable contributions to educational literature. 
Historical research is discussed in Chapter VI. 

2. Normative-survey research, This method may utilize a 
number of tools and procedures, such as questionnaires, tests, 
check-lists, rating scales, score cards, interviews, etc. For this 
type of investigation several specialized procedures have been 
developed: for making comparisons (between periods in his- 
tory, nations, regions, states, counties, cities, districts, schools, 
grades, departments, subjects, school enterprises etc.), for re- 
vealing status or central tendencies, for making predictions of 
future performances, and sometimes for determining causes 
of present conditions. These procedures, for the most part 
normative in character, may be classified as established phases 
of the normative-survey technique. The national surveys of 
secondary education and teacher training, already listed in 
Chapter III, the Modern Foreign Language Study, the Educa- 
tional Finance Inquiry, and the educational year-books of the 
International Institute of Teachers College, Columbia Univer- 
sity, are good examples of this method. Testing surveys prop- 
erly belong in this category. 

Techniques of analysis (as applied to activities, jobs, social 
needs or usage, books, courses of study, difficulties, etc.) have 
been borrowed from the industrial world to aid in determining 
what activities to emphasize most profitably in the school. 
These methods have been applied frequently in analyzing 
social need or usage as a guide in curriculum’ reorganization. 
They usually result in central tendencies of the normative- 
survey type. The normative-survey method is discussed in 
Chapters VII and VIII under six types of studies employing: 
(a) survey-testing, (b) questionnaire, (c) documentary-fre- 
quency, (d) interview, (e) observational, and (f) survey- 
appraisal procedures, including school surveys. 

3. Experimental research. The experimental method also 
is considered a major type of educational research. The ex- 
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perimental technique, whether used in classroom or laboratory, 
has possibilities for solution of numerous and significant learn- 
ing and teaching problems through the process of controlled 
observation and measurement. Of course the greatest difficulty 
in using this method with human beings is the control of vari- 
able factors. In experimental work, the logical principle of 
“difference” is employed, since with all factors save one con- 
trolled and equated for subjects or groups, differences appear- 
ing as a result of the operation of the single variable are noted. 
The experimental method, including the control of variable 
factors, is discussed in Chapter IX. 

The laboratory method almost always involves a type of 
experimental approach, although usually it is a more intensive 
study of a smaller number of subjects than classroom experi- 
mentation where the group technique prevails. Obviously, it is 
a special form of the experimental method, and is discussed in 
Chapter IX. 

4. Other methods of research especially adapted to studying 
complex causal relationships. In certain other types of investiga- 
tion one observes an effect and seeks to determine its cause ; for 
example, in the case of a teacher who has done poorly in her 
instructional duties, a pupil who has failed in one or more 
school subjects, or a class which has made relatively little 
progress in attaining its educational goals.*° 

The investigator may concentrate his attention on single 
instances of poor work, conducting a diagnostic type of an- 
alysis similar to intensive case-study procedure, or he may seek 
to determine the causes of poor performance common to a 
number of instances. Research concerned with such causal- 
comparative relationships involves the logical principle of 
“agreement” in that situations are compared to discover like- 
nesses or common factors which may produce, let us say, good 
teaching or poor teaching. Case-study procedures, diagnostic 
and prognostic instruments, and correlation techniques have 


30 A. S. Barr, An Introduction to the Scientific Study of Classroom Supervision, 
Chapter IV. New York: D. Appleton and Co.. 1931. 
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been used extensively in this type of research, which is dis- 
cussed in Chapter X. š 

The case method has been borrowed from medicine, law, 
sociai service, and military science, and applied in education 
to problems of abnormality, brightness, dullness, adjustment. 
adolescence, problem children, special talents and defects, 
mental and physical growth, etc. Although the intensive an- 
alysis of many individual cases may result in the establishment 
of central tendencies and generalizations as in the survey or 
normative method, the case technique because of its emphasis 
on diagnosis seems to merit treatment as a causal type of 
investigation which employs both documentary and non-docu- 
mentary evidence written up in the form of case histories. 

The genetic method is developmental in character, involving 
a series of observations (often uncontrolled), or cross sections. 
In this type of research causal factors frequently are investi- 
gated. Genetic studies are similar to normative-survey investi- 
gations in the sense that norms of development or attainment 
are involved. They are of an experimental character when con- 
trolled conditions of observation and measurement are estab- 
lished. 

Both case and genetic methods involve a type of historical 
approach. Excellent examples of both procedures are found in 
the Stanford Genetic Studies of Genius. Gesell’s and Bald- 
win’s growth studies are outstanding illustrations of the genetic 
type of research.** Case and genetic methods are discussed in 
Chapter X. 

Distinguishing characteristics of the major research 
methods. This fourfold classification may be presented to ad- 
vantage in tabular form, with the sources of data, type of 

31L. M, Terman and Others, Genetic Studies of Genius, Vols. I, I, III. Stanford, 
Calif.: Stanford University Press, 1925, 1926, 1930. Pp. 648, 842, 508. 
cen Gesell, Infancy and Human Growth. New York: The Macmillan Co., 1928. 


Arnold Gesell and Others, An Atlas of Human Behavior, Vols. I and II. New Haven, 
Conn.: Yale University Press, 1934. Pp. 922. 


B. T. Baldwin, The Physical Growth of Children from Birth to Maturity. University 
of Iowa Studies in Child Welfare, Vol. I, No. 1. Iowa City, Iowa: University of Iowa, 


1921. Pp. 412- 
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control, approach, and typical purposes indicated for each re- 
search method. The chief sources of data are direct and indirect 
observation, with the aid of various measuring instruments, 
data-gathering devices, and documents. Observation may be 
under controlled conditions, as in experimental and laboratory 
studies, or uncontrolled, as in historical and in most survey 
studies. The approach may be longitudinal, with a fairly long 
time span, as in historical and in most genetic and case-study 
research (with a limited number of subjects in genetic and 
case investigations), or cross-sectional, as in normative-survey 
and experimental work (with sampling of a fairly large num- 
ber of cases over a relatively short period of time in normative- 
survey studies). It will be noted that any of the four methods 
of research may be employed to determine causal relationships, 
although this is not the primary purpose of normative-survey 
studies. Some interesting comparisons may be noted in the last 
column with respect to the typical forms in which the results 
of the several types of research are stated. 

Devices, tools, or aids in the collection, analysis, and inter- 
pretation of data. There are certain procedures not of them- 
selves fundamental research methods in the search for truth, 
but which rather serve as tools or devices in collecting or 
analyzing data. The use of questionnaires, check-lists, rating 
scales, and interviews is discussed in Chapters VII and V Ill. 
The questionnaire is a useful, if overworked and abused, device 
for securing educational data. The interview is in part an oral- 
questionnaire technique. Check-lists, rating scales, and score 
cards may be forms of the questionnaire. 

Statistical techniques are discussed in still another chapter 
(XI) which deals with the analysis, classification, and sum- 
marization of data. The statistical method may be considered 
as a way of manipulating data rather than as a technique for 
securing new information. Of course there are a few pieces of 
research in which new formulae, machines, or statistical de- 
vices, are worked out, for example, a statistical slide-rule, 
machines for testing and for drill, or a new method for obtain- 


sumou qymorg 


S104 
S S23 Penpapur 


sovu p 
pue səssouəyı dnog 


sds 
-uone pesneo jo uoppeuurpq 
spag| adv quaregip 7e sou 

PUE 83e ISHAILILYI JO ÁIIAOISI 
quauidopaap pue 

mo ‘efueyo jo uoneuunaəq 


sdiqs 
-uonepa ywsNvd jo uopeurnopaq 


uoporpozd pur sdiys 
-uoe pesnes jo uovu 

sdqs 
-uogepa pesneo jo uoneupwapq 


j; 

Jeuonsas | 
“$019 

aq Aew mq 

«xqqeaajaid 

Jeurpnysu0T 


[eurpny1su07y 


Jeuorjses 
-85019 
reuonəs 
-s5039 


worwazasqo pajon 
-uoun pue pəjjorzuog 


uoreaIas 
-q0 pajonuoomn Apa9 

wor RAIas 
-qo payjorjuosun ÁI 


uopeazasqo payjoruesuy, 


yuəwarnsvəw 

pue uopearsqo parq 
aouaptaa Areyuauinsed 
pue ‘quaurainsvaw ‘uova 
~195q0 yaxTpUL pue PLL 


quəunsvəu 
pue uopearsqo JNA 

quawamsraut 
pue uonearasqo parq 


NUD (P) 


Apnys aseQ (2) 


wor Lpex109 (4) 
aanereduos 
— sneg (9) 
sdiqsuor 
~eja esnuo xəjd 
-uo9 3urpn}s 
o} peydepe 
Alre roədsə 
spogpw 90 t 


səñweqə poonpord 
uəwədxə Jo sə 
I put syuncwy 


semnpaa 
oid wuorwonpa jo Ayjejuawitred 
-xə uosueduros put woryenyTAa 


yeuornas 
-55019 


worwAzesqo payjonwoD 


+ quaurensvour 


“oyeyuawmLdxg ‘E 


AN 
-vwa pue Aduopud} 
pua jo semnsvoyy 


('Poajoaur aq Aver 
sdiysuorvjax Jesnv> sasvd əwos u) 

auy quaserd ay} 7X suoT}IpUOD 
pur s}9fqo Jo uostzeduios pure snes 


Burjs9q Jo aseo ayy Ut 
payjorquos anq ‘uoneasos 
-q0 payjonuosun Ág 


*979‘systj-yooys‘saatwuu0ly 
-sanb ygnosy3 ‘uonvaIəs 
-qo yampuy ‘quawems 
-eau pre maəparagur Iy} 
UL se ‘uopearasqo PANA 


**Aaams 
DAwULION *z 


woryeqaidsaqur 
pur uorsodxa equa, 


sdiysuorrjar msneo Jo uonevu 
soimpooord pur suiajqord Avp-quasaid 
Jo uonenpeaa pue uorjezordoquy 
Səung UoAI3 4e vuəwouəyd Jo sneg 

squsas ysed Jo p10391 3JEMIY 


Teerpny suo] 


uoneazasqo pajjonuovun 


‘SOSSOUIIA 
242 pur ‘sureurar ‘syuou 
-nəop yZnorq} uoreazas 
-q0 qoexpur fssouzrat-240 
UE se uonearsqo nq 


APENGA yeoq03stgy r 


synsoy 3 
10} SW 


sosoding PANAL 


ypvoiddy 


1013u00 jo adh], 


req JO amog 


PPN 


SAORI BJAVISTY BOLV]Y TAL 40 SOLSTHALVUVHD ONIHSINONIISI INOS 


230 EDUCATIONAL RESEARCH 


ing correlation coefficients. Test construction and testing 
studies involve extensively the following statistically derived 
characteristics: central tendency, variability, reliability, valid- 
ity, and relationship. The appropriate place for discussion of 
tests and test construction is in Chapter VII, along with 
certain other data-gathering devices. 

It is apparent that some of these data-collecting devices 
provide quantitative and objective data—for example, the 
salaries of teachers or the average daily attendance of pupils, 
others secure qualitative and subjective information—for ex- 
ample, the rating of a teacher, a textbook, or a school building. 
In analyzing data secured through such media, it is necessary 
in some cases, especially qualitative or subjective studies, to 
use verbal methods, where statistical procedures and terms do 
not apply. However, these are problems for consideration in 
the appropriate subsequent chapters (XI and XII). 

Comment should be made concerning the disposal of so- 
called philosophical techniques in this book. Many students of 
objective methods in educational research do not concede the 
existence of a philosophical technique of investigation. How- 
ever, it is to rationalization, good thinking, and sound logic 
that one must turn in the interpretation of data and in de- 
termining ultimate goals and values. Even though this tech- 
nique of thinking is not accepted everywhere as a type of 
research, it must be admitted that subjective technique has led 
to important contributions to educational literature. 

Will Durant has expressed strikingly the relationship be- 
tween science and philosophy, by saying that mere facts mean 
little except in relation to desire, purpose, a whole, and in 
terms of perspective and valuation.’ “Science gives us knowl- 
edge, but only philosophy can give us wisdom.” If science is 
the foundation of the cathedral of learning, philosophy isthe 
spire which points onward and upward. : 

Space is not available in this chapter for a more detailed 
statement of the place of philosophy in educational research. 


33 Will Durant, The Story of Philosophy, p. 3. New York: Simon and Schuster, 1926. 
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The authors find its chief function in the interpretation and 
evaluation of data, as well as in the formulation of hypotheses. 
Therefore, the contribution of philosophical and logical proce- 
dures is discussed in other chapters (IV, XI, and XII) which 
deal with problems of setting up hypotheses, critical analysis 
of evidence, and formulation of conclusions and generalizations. 
At this time, the interested reader should examine current dis- 
cussions of the place of science and philosophy in education.** 
Burton in particular has emphasized the close relationship be- 
tween philosophy and science, although he devotes separate 
chapters to “The Science of Education and Its Method” and 
to “The Philosophy of Education and Its Method.” These com- 
plementary and supplementary aspects of philosophy and 
science as phases of problem solving or as steps in reflective 

34W. H. Burton, Introduction to Education, Part II. New York: D. Appleton-Century 
Cog 29545. on, “Orientation of Educational Research to Present-day Science and Phi- 
losophy,” Educational Research Bulletin, XIIL (February 14, 1934), 29-38, 54. 

H. A. Clugston and R. A. Davis, “Is a Scientific Method Possible for Philosophical 


Research in Education?” Educational Administration and Supervision, XVI (April, 1930), 
293-99. 


„ “Suggested Criteria for the Philosophical Method of Research in Education,” 
Educational Administration and Supervision, XVI (November, 1930), 575-80. 

Michael Demiashkevich, An Introduction to the Philosophy of Education. New York: 
American Book Co., 1935. Chapter I deals with the science and the philosophy of 
education. 

John Dewey, “The Need for a Philosophy of Education,” New Era, XV (November, 
1934), 211-7. 

R. L. Finney, ‘Philosophy versus Science Again,” Educational Administration and 
Supervision, XVI (March, 1930), 161-73. 

F. N. Freeman, “Scientific and Philosophical Methods in Education,” Science, 
LXXIII (January 16, 1931), 54-9. 

H. W. Holmes, “The Philosophic Element in Scientific Method,” School and Society, 
XLI (January 12, 1935), 41-8. 

H. G. Hullfish, “The Relation of Philosophy and Science in Education,” Journal of 
Educational Research, XX (October, 1929), 159-65. 

T. L. Kelley, ‘The Scientific versus the Philosophical Approach to the Novel Prob- 
lem,” Science, LXXI (March 21, 1930), 295-302. 

W. H. Kilpatrick, “Philosophy and Research: The Relations of Philosophy and 
Science in the Study of Education,” School and Society, XXX (July 13, 1929), 39-48. 
, “The Relation of Philosophy to Scientific Research,” Journal of Educational 
Research, XXIV (September, 1931), 97-114. 

M. A. May, “The Relative Value of Science and Philosophy in Appraisal,” The Direct 
Contribution of Educational Psychology to Teacher Training, Chapter VIII, Yearbook 
XX of the National Society of College Teachers of Education. Chicago: University of 
Chicago Press, 1932. 

R. B. Raup, “Limitations of the Scientific Method,” Teachers College Record, XXX 
(December, 1928), 21226. 

R. E. Waitt, “A Piea for a More Appreciative Codperation between Scientist and 
Philosopher in Education,” Education, XLIX (May, 1929), 513-21. 

Frederick L. Whitney, “Some Limitations of Scientific Method in Education,” Teacher: 
Journal and Abstract, IV (January, 1929), 59-62 
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thinking were pointed out near the end of Chapter I, and have 
been analyzed elsewhere in more detail by one of the present 
writers.*° 

Concluding statement. The reader is reminded again that 
no single classification of research methods is satisfactory for 
all purposes and certainly not for all investigators. However, 
the simple working classification adopted for purposes of dis- 
cussion in this volume seems usable and will help the student 
to attack problems or collect data for the solution of prob- 
lems, as well as show the steps of problem solving. It has been 
said in this chapter that in attacking most of our educational 
problems—the adaptation of instruction to varying needs, for 
example—there are probably four major approaches: (1) to 
examine the evidence and experience of the past as an 
aid in analyzing and interpreting the present situation 
(historical research); (2) to canvass present practice with 
respect to plans for dealing with such differences or to 
set up norms or central tendencies through testing and meas- 
urement against which the extent of differences may be 
checked (normative-survey research); (3) under controlled 
conditions, with only a single variable, to try out in classroom 
or laboratory different ways of dealing with or adjusting to 
individual differences in pupils in order to determine the 
effectiveness of a given plan or procedure (experimental re- 
search); (4) to determine relationships through statistical 
manipulation of data (correlational calculations for example) or 
through intensive case or genetic study or causal-comparative 
group analysis, to diagnose the origin or cause of differences 
and possibly to follow such analyses by remedial prescription 
and practice (types of research especially adapted to studying 
complex causal relationships). Likewise, devices for collecting 
data (such as the questionnaire), statistical techniques for 
analysis and interpretation of data, and so-called philosophical 
methods (useful in formulating and testing hypotheses, and in 


a5 A, S. Barr, “Science and Philosophy in Supervision,” Education, LIII (June, 1933), 
887-93. 
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interpreting data), have been assigned appropriate places in the 
pattern or steps of problem solving and scientific method. 


PROBLEMS AND EXERCISES 


t. Select some classification of research methods, and criticize it in 
terms of comprehensiveness, internal consistency, and possibilities 
for attack on educational problems. 

2, Show how the classification chosen in connection with the pre- 
ceding assignment may be reorganized so as to agree reasonably 
well with the pattern of research methods which this book repre- 
sents. 
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CHAPTER VI 


THE HISTORICAL METHOD 


Introduction. This chapter considers the contribution made 
by the historical method and by educational history, the scope 
and frequency of historical studies in education, collection of 
historical data, criticism of such data, and the writing of 
history. Special attention is given to legal and bibliographical 
or summary studies because of their current interest and sig- 
nificance in education. 

In dealing with historical research and educational history in 
this chapter, no attempt is made to draw a sharp line between 
historical method as a research procedure and the history of 
education as a field of professional knowledge. The former pro- 
vides a technique of investigation which makes possible the 
existence of a body of content in the history of education; for 
purposes of the present, or any similar, discussion there need 
be no divorce between method and content. 

The purpose of the educational historian may be either to 
produce a faithful record of unique events that have happened 
in the past or to suggest through the survey of these events 
fruitful generalizations from past experiences that may act as 
controls for behavior in the present or future. The educational 
historian may choose either or both of these objectives as the 
guiding purpose in his research. In the survey of previous ex- 
periences, investigations, and critical thought with respect to 
some selected current problem or issue, one is engaged in his- 
torical research where the purpose is to profit by the experi- 
ences of the past in the solution of present-day problems. 
Unfortunately, this functional character of historical research 
is frequently overlooked. 

The functional purposes of historical research in education 
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are stressed repeatedly in this chapter in showing how his- 
torical materials may explain major types of educational or- 
ganization, activities, and problems of to-day. It is pointed out 
that: (1) a study of the evolution of nationalism and democ- 
racy, and of the thinking of great leaders, may contribute to 
an understanding of contemporary problems; (2) educational 
history aids in the detection of fads and frills, acts as a sov- 
ereign solvent for pedagogical prejudices, and reveals how edu- 
cation as a social institution has become centralized and 
complex, as contrasted to an earlier form of decentralized and 
local control of schools. A plan is described for making his- 
torical research function in schools through well-kept records 
of actual teaching procedures and case studies of the pedagog- 
ical growth of pupils. A detailed illustration shows how his- 
torical methods throw light on one of the most vigorously 
discussed current problems, the activity school or progressive 
education. The legal type of research in education is an espe- 
cially concrete application of principles of historical research, 
dealing with such problems as the right of the teacher to con- 
trol children going to and from school, the teacher’s liability 
for injury to children while at school, the influence of the 
court on the curriculum, etc. Finally, the thorough bibliograph- 
ical survey of earlier studies in a given field, accompanied in 
many instances by critical analysis and interpretation of the 
findings, is an especially appropriate area for functional appli- 
cation of the principles of historical research, as pointed out 
later in this chapter. 

Since the fundamental procedures in historical investigation, 
with respect to collection of data, criticism of data, and inter- 
pretation, must be observed, whether one’s purpose is merely a 
faithful record of unique events of the past or the solution of 
current problems through the experiences of the past, it seems 
wise to turn for guidance to major treatises on the historical 
method, prepared by the historians themselves. Although some 
of these historians stress the “record” aspects of their research, 
as is clear in succeeding paragraphs, the reader will find ap- 
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propriate emphasis placed on the functional elements of educa- 
tional history in other parts of this chapter. The educational 
historian is not so much interested in the purely unique aspects 
of past experiences as in the elements which may serve as a 
basis for tentative generalizations in analyzing current issues 
and problems. The limitations of history as a basis for prob- 
lem solving will be considered as a part of the subsequent dis- 
cussions of collection of data, criticism of data, and interpre- 
tation. 


NATURE, VALUE, AND SCOPE OF HISTORICAL RESEARCH 


Nature of historical research. History denotes any effort te 
recount any aspect of the past life of mankind, such as war, 
diplomacy, art, institutions, travel, science, industry, biography, 
or thought, Therefore, the possible field of historical research 
and writing is as broad as life itself. More specifically, one 
may be interested in the history of politics, school legislation, 
the family, science, intelligence testing, the school survey, the 
activity school, visual instruction, the project method, whole 
versus part methods of learning, or laboratory technique in ex- 
perimental education. 

Three major processes are involved in historical method: ( 1) 
collection of data, with consideration of documents and relics, 
of primary and secondary sources, of bibliographical procedure, 
and of organization of materials; (2) criticism of data col- 
lected, including the processes of external (or lower) criticism 
and internal (or higher) criticism; and (3) the presentation of 
the facts in readable form, involving problems of organization, 
composition, exposition, and interpretation. 

History differs from the natural sciences because it is not 
based upon experimentation, but upon reports of observations 
which cannot be repeated, although similar events may occur. 
It is this interest in the unique which makes it impossible for 


1H. C. Hockett, Introduction to Research in American History, p. xi. New York: 
The Macmillan Co., 193% 
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the historian to take advantage of experimentation; he cannot 
conjure up the figures of the past and cause them to reproduce 
the famous scenes of history.? The historian has rarely wit- 
nessed the original event, but from documents and remains he 
attempts to reconstruct the historical past. Of course, the user 
of the experimental method may study directly changes taking 
place and may repeat and verify as often as desired the con- 
ditions under investigation, controlling the factors involved. 
Normative-survey research emphasizes analysis and eyalu- 
ation of current rather than past events, and predicts occur- 
rence of such events as the height, weight, or achievement in 
reading of school children at a given age or time. 

Value and use of historical research in education. It has 
been suggested that the prestige of the historical method has 
suffered in education because of the early availability of his- 
torical materials and, therefore, the requirement of history of 
education in the early teacher-training programs in this 
country. In most cases this work was given in a standard form, 
involving the rather mechanical study of a mass of compara- 
tively unrelated historical facts, without application to the 
social situation or to the interests, activities, and problems of 
educational workers. 

In outlining a history of education designed to take the place 
of this older form of training, Reisner shows how a study of 
the evolution of nationalism and democracy, as well as of the 
thinking of great leaders, may contribute to an understanding 
and evaluation of present-day educational problems and of 
significant factors in complicated modern society.* 


It may be said that the historical study of education does not 
present us with ready-made and perfect conceptions which may enter 


2F, M. Fling, The Writing of History, pp. 23-4. New Haven, Conn.: Yale University 
Press, 1920. 

Allen Johnson, The Historian and Historical Evidence, p. 45. New York: Charles 
Scribner’s Sons, 1926. 

3 Edward H. Reisner, “The History of Education as a Source of Fundamental Assump- 
tions in Education,” Educational Administration and Supervision, XIV (September, 1928), 

8-84. 
Hiio see R. H. Wheeler, “A Set of Postulates for Educational Theory, I: The 
Background,” Journal of Educational Research, XXVII (January, 1935). 321-33. 
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without modification into our educational thought. Neither is it 
claimed that the mere data of educational history fall of their own 
weight into thought patterns. It is submitted that, in so far as educa- 
tion is a social process, comprehensive and realistic thinking about 
education must depend upon a knowledge of the origins which influence 
its present state. It is submitted further that the serial approach 
makes easier the recognition and identification of significant factors 
in the bafflingly complicated present situation, It is submitted, finally, 
that a critical estimate of the partial viewpoints and the keen insights 
of educational thinkers who have written their names into the his- 
tory of Western civilization not only may protect us against the too- 
easy acceptance of half-concepts which bid for our approval, but also 
may reveal to us the organic and comprehensive meaning of education 
which the future may make into a reality. 


Knight’s excellent statement of the possible contribution of 
historical research in education and, by the same token, of the 
history of education, is as follows: * 


1. A knowledge of the history of schools and other educational 
agencies is an important part of the professional training of the 
teacher or the school administrator. 

2, Much of the work of the school is traditional. The nature of the 
work of the teacher and the school administrator is restrictive and 
tends to foster prejudices in favor of familiar methods. The history of 
education is the “sovereign solvent” of educational prejudices. 

3. The history of education enables the educational worker to 
detect fads and frills in whatever form they may appear, and it serves 
as a necessary preliminary to educational reform. 

4. Only in the light of their origin and growth can the numerous 
educational problems of the present be viewed sympathetically and 
without bias by the teacher, the school administrator, or the public. 

5. The history of education shows how the functions of social 
institutions shift and how the support and control of education have 
changed from very simple and local arrangements to those that are 
now somewhat centralized and complex. 

6. The history of education is an ally in the scientific study of 
education rather than a competitor. It serves to present the educational 
ideals and standards of other times, and it enables social workers to 
avoid the mistakes of the past. 

7. It inspires respect for sound scholarship and reverence for great 
teachers. 


4E. W. Knight, Education in the United States, p. 38. Boston: Ginn and Co., 1934. 
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Other good analyses of the value of historical research have 
been made by Abelson * and Alexander.® 

Attempts have been made to meet the previously mentioned 
criticisms of earlier courses in the history of education, in order 
that the historical method and the body of resulting materials 
may serve functional purposes and social needs. Tyler has de- 
veloped an organization for a course in this field which pre- 
serves such merits as a sequential study of chronological peri- 
ods may possess and at the same time makes possible the use 
of historical materials to explain major types of educational 
organization and teaching activities." 


The plan of teaching is graphically shown in Table XI, where it 
may be seen that the same content can be included both in the 
organization according to the horizontal units, largely chronological, 
named and lettered at the left of the table and in the organization 
named and numbered in the vertical columns, Although the historical 
material selected for the course was that which helped to explain 
the major types of educational structure and teaching activities, the 
material was first presented to the students in accordance with the 
units named at the left of the table, In the first nine weeks of the quar- 
ter these units were considered one at a time so that the students 
might get the sequence and story of educational development. Class 
exercises during this period involved the selection of forces which 
were noticeably influencing education in the past and which have 
potent influence to-day, as well as requiring students to organize the 
main points of the story of educational development. [This plan is 
more commonly followed than that described in the next paragraph. ] 

The last three weeks of the quarter were employed to provide an 
Dpportunity for the students to reorganize the material. In this period 
the specific items of educational structure, practice, and theory were 
grouped into the twelve units shown in Table XI,...and the class 
exercises called for an organization of historical material so as to 
explain these items. This required a reorganization of content already 
considered in the previous period and at times a collection of addi- 
tional material by the student when his previous information failed to 
provide an adequate explanation. 


5H. H. Abelson, The Art of Educational Research, pp. 85-8. Yonkers-on-Hudson, 
N. Y.: World Book Co., 1933. 

eCarter Alexander, How to Locate Educational Information and Data, pp. 210-1, 
New York: Bureau of Publication, Teachers College, Columbia University, 1935. 

7R. W. Tyler, “A Course in History of Education,” Educational Research Bulletin, 
IX (February 5 and March 5, 1930), 57-65, 133-5- 
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Scope and frequency of historical studies in education, In 
order to illustrate the range of historical problems in education 
investigated within the past few years by graduate students, 
many of whom are field workers, the titles of a number of theses 
may be mentioned. Fuller information concerning these studies 
may be found in the annual compilations of research published 
by the Office of Education, under the headings of educational 
history and educational biography.® Illustrative titles are: 


1. “An Historical Study of Otterbein: College at Westerville in 
the State of Ohio.” 
2, “History of the Founding of Educational Institutions by the 
Disciples of Christ in Virginia and West Virginia.” 
3. “The Development of the Arizona Public-School System.” 
4. “History of Catholic Education in Kansas, 1836-1032.” 
5. “Secondary Education in Georgia, 1732-1858.” 
6. “A History of Catholic Colleges for Women in the United 
States of America.” 
. “The Educational Contributions of the United States Commis- 
sioners of Education, 1867-1928.” 
8. “The Public-School System and the Second Constitution of 
California.” 
9. “Dominant Factors in the Development of Public Education 
in Kansas.” 
ro. “Analysis of Educational Studies Made under the Auspices 
of the Association of American Universities from 1910-1931.” 
11. “The PrivatesAcademy Movement in Indiana from 1850 to 
1900,” 
12. “Genesis and Growth of the Progressive Movement in Edu- 
cation.” 
13. “Education in Hawaii, 1820-1893.” 
14. “Legislation Affecting Secondary Education in Virginia from 
1619 to 1845.” 
1s. “A Critical Study of Certain Aspects of Infant and Common 
School Education, 1810-1840.” 
16. “An Historical Study of the Methods of Teaching as Used 
by Eminent University and College Teachers.” 
17. “State Constitutional Development in the United States, 1829- 
1851.” 
8 For example, Ruth A. Gray, Bibliography of- Research Studies in Education, 
1932-1933. Office of Education Bulletin, No. 7, 1934. Washington: Office of Education, 


1934. Pp. xiv + 350. 
, Bibliography of Research Studies in Education, 1933-1934. Office of Education 
Bulletin, No, s, 193s. Washington: Office of Education, 1935. Pp. xiv + 328. 
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18. “Mother Seton—Foundress of the American Catholic Parochial 


School System.” 
19. “The Life and Work of Johann Bernhard Basedow: us 


TABLE XI 
Two METHODS FOR ORGANIZING CONTENT IN History oF EDUCATION 


By the Major Types of Educational 
Structure and Teaching Activities Which 


By Units Which Are Largely 
Ghtcnclowcaliberods the Historical Material Helps to Explain 
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It will be noted that the last two of the foregoing titles 
represent biographical studies while a number are investiga- 
tions of constitutional or legal aspects of education, to which 
extended discussion is devoted later in this chapter. The meth- 
ods of historical research, however, are fundamentally the 
same, whatever the type of problem studied. 

The historical method of research was employed with 
greater relative frequency in education twenty or more years 
ago than at present. It was to be expected, with the develop- 
ment of experimental, survey, testing, and statistical procedures 
in education, that educational history and philosophy would 
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have to surrender a large part of the field which they had 
monopolized for some time. However, if the use of the historical 
method has decreased, legal research in education, which re- 
sembles historical investigation very closely and may be con- 
sidered a phase of historical:research, has shown a correspond- 
ing increase in popularity over the same period of time. In 
fact, the attention of some workers who formerly gave their 
time to research in the history of education has been diverted 
to a study of the specialized problems of educational legisla- 
tion and court decisions affecting the school, as witnessed by 
the thesis titles quoted above. This development seems to jus- 
tify a discussion of historical and legal research in education 
under the same chapter heading; both procedures involve 
painstaking documentary study of sources, criticism of data, 
and interpretation. 

Concrete evidence concerning the frequency of the historical 
and legal methods is available.” Table XII reveals, in so far 
as can be determined from an examination of the titles of 
theses, to what extent historical or legal techniques of re- 
search have been used. The facts reported show that approxi- 
mately one-ninth of all Doctors’ theses completed and under 
way, 1918-1931, involve either the historical or legal technique 
of investigation. The fields which seem, on the basis of fre- 
quency, most in need of historical studies are psychology and 
measurement, and elementary education, although it is recog- 
nized that infrequency may not always be a valid criterion 
of needed research in a given area. 

Loomis suggests two important types of historical research 
in the curriculum area, utilizing sources other than textbooks 
and courses of study, namely, well-kept records of actual teach- 
ing procedures and case studies of the pedagogical growth of 
pupils. The University of Chicago Laboratory Schools have 
been keeping such detailed records over a number of years, 
and at intervals reports on teaching procedures in given school 


9 Carter V. Good, “Fields and Types of Research in Education, 1918-1931,” Journal 
of Educational Research, XXIV (June. 1937). 33-43- 


TABLE XII 


CLASSIFICATION OF 151 DOCTORS’ THESES IN EDUCATION COMPLETED, 1918-1929, AND OF SEVENTY-TWo DOCTORS’ THESES 
UNDER WAY, 1930-1931, EMPLOYING HISTORICAL or LEGAL TECHNIQUES 


1918- |Under| 

29 | Way |Grand 

Field 1918|1919|1920|1921|1922|1923/1924) 1925|1926|1927|1928|1929| Total |’30-’31| Total 
Administration and supervision. ......... 2 ie 2 I I EBA 2a tee tall ee ee Ue Or rar 19 6o 
Psychology and measurement (general). CM AE E cor e E A E E ilo, oF 3 I 4 
General method and theory hae 2 I oj o I I 2 3 v E- a ee) 18 9 27 
History of education and foreign education.| 13 | 8| 7| 3| 7| 12} 10] 14] 15 | 22 | 17 19 | 147 53 | 200 
Preschool and elementary education . HEO E Oah Fo) lars DRAN so.) cree el top lier dee a 5 1 6 
Secondary education............. 3 ran A OE: Ta E & I I 2 TI Slaa i a) aay Io 27 
Higher education and adult education.....| x I iT |= Ost x 20) ae] 40: 1538) A E a -A =x8 IL 29 
Schoolisubjects 220. cee e A ae. ae EU OM MS A he ate ea | att Sah 6a] gyineg | 4 |=a9 3 32 

Total theses... 13| 8| 6| 4] 8| 13] 10] 16 | 21 | 23 |18 | 1x | x51* | 72* | 223* 


“This, the total number of historical theses, is less than the total times the various fields are represented, since a given thesis may be classified under 


more than one subdivision of education. 
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subjects are appearing, the plan being to cover each field once 
in three years.?° 

In the appendix is a selected list of problems considered 
worthy of investigation in the history of education, although 
they are rather large in scope for the beginner in historical re- 
search, who may do better by selecting a smaller topic such as 
the history of the local high school. Criteria for the selection 
of a problem have been discussed in Chapter II, and the train- 
ing of the research worker is considered in Chapter XV. 

Although no supporting data in terms of frequencies are 
immediately available at this writing, the authors have ob- 
served that many competent scholars sense the futility of 
many of the superficial surveys and questionnaire studies and 
are turning again to educational history, as well as to experi- 
mentation, for aid in solving their problems. However, this 
statement must not be interpreted to mean that different types 
of research compete with each other. Selection and evaluation 
of a method of investigation should be determined by purpose. 
Obviously, educational history cannot tell a superintendent 
how other school systems are making adaptations to a period 
of economic depression and social disturbance, although it 
can show how such cycles of stress have been weathered in the 
past. As a practical pursuit, with functional purposes and 
social need to be served, the history of education seeks to 
answer three rather different types of questions: +° 


1, The question of evolution. How did progressive education arise 
and develop? We seek to establish trends. 

2. The question of resemblance. How is education in Denmark like 
that of Italy, and how unlike? We institute comparisons. 

3. The question of value. Why should the chief state school officer 
be chosen by a lay board rather than by popular election? We find 
causes and reasons. 


10A, K. Loomis, “Needed Research in the Curriculum,” Journal of Educationa 
Research, XXIX (October, 1935), 126-30. 

11H. G. Good, “The Possibilities of Historical Research,” Journal of Educational 
Research, XXIX (October, 1935), 148-53. 

12 Ibid., p. 149- 
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COLLECTION OF DATA 


No matter what the type of research, after the problem has 
been selected, defined, and delimited, the next step) is to collect 
data pertifient to the solution of the problem. In historical 
research the classification of source materials is important, 
although most existing classifications are open to criticism. 

Historical sources. Vincent divides historical materials into: 
(1) consciously transmitted information, including (a) written 
sources (chronicles, annals, biographies, memoirs, diaries, 
genealogies, and certain classes of inscriptions), (b) oral tradi- 
tion (ballads, anecdotes, tales, and saga), and (c) artistic 
productions (historical paintings, portraits, scenic sculpture, 
and coin types); (2) relics or unconscious testimony (human 
remains, language, institutions, products of the hand, imple- 
ments, fine arts, literature, etc.) ; and (3) inscriptions, monu- 
ments, and public documents of certain types, all of which may 
possess characteristics belonging to the first two types of 
sources.* This third group has sometimes been labeled 
“memorials.” 

Ordinarily it is not difficult to distinguish between the 
written records or documents, which man has prepared with 
the deliberate purpose of transmitting information, and relics 
or remains of man’s activity (written or otherwise), which 
have been handed down from the past without the conscious 
intention of imparting information. However, even in difficult 
cases, it is essential to classify historical materials in terms 
of conscious intent to leave recorded information or of uncon- 
scious. testimony in the form of relics or remains. 

Vincent stresses the fact that differences between “memo- 
rials” and transmitted testimony are not always immediately 
apparent, but must be determined as a vital part of historical 
research.'* He suggests that a tombstone containing only the 
name of a man is a relic of the past, but that when dates are 


13 J. M. Vincent, Historical Research, p. 18. New York: Henry Holt and Co., rorr. 
14 Ibid., Chapter II. 
F. M. Fling, cp. cit., pp- 44-6. 


THE HISTORICAL METHOD 251 


included, the historical record begins, and when the epitaph 
recounts the deeds and virtues of the deceased, the relic be- 
comes a memorial with the characteristics of a document or 
consciously transmitted information. 

Another writer (Fling, see footnote 14) uses the term “tradi- 
tion” to cover all materials containing the impressions made 
upon individual minds by man’s past activities, whether in 
written, oral, or pictorial form, which includes the same area 
as Vincent’s first major division cited earlier. 

Certain difficulties have been encountered in seeking to learn 
from recorded experience.” 


The record is incomplete and otherwise defective from several 
causes. The making of records requires time, money, and wisdom. So 
does their preservation. The records of one of the most illustrious 
of all our state school officials were taken out of the State House 
by a successor in the office and burned—perhaps, because they were 
out of date! And records must be made available if they are to be 
useful, Other factors are involved. Each age records: (1) what it 
considers to be of interest or value, (2) what it has the eye to see, 
and (3) the will to make known. No age is able to anticipate very 
completely the questions to which following ages will seek the answers. 
We cannot make new records of past events and conditions, and there- 
fore we must learn from such records as we have. 


Sources in educational history. Probably the most useful 
and pertinent classification of historical sources in education is 
twofold: reports of events (called documents) and physical 
objects of historical value (called remains or relics). 

In an excellent article H. G. Good points out that “remains” 
in the history of education are numerous, but have never re- 
ceived adequate attention.** He considers as remains: school 
buildings and their furnishings; photographs of buildings or 
furnishings, or of children, teachers, and parents engaged in 
educational activities; forms of diplomas, attendance, and cer- 
tificates, and record blanks; various physical devices of the 

15H. G. Good, “The Possibilities of Historical Research,” Journal of Educational Re- 


search, XXIX (October, 1935), 148. fz 
16H. G. Good, “Historical Research in Education,” Educational Research Bulletin. 


IX (January 8 and 22, and February 5, 1930), 7-18, 39-47, 74-8. The authors are much 
indebted to these articles in the present discussion. 
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school for teaching, punishment, exercise, or health; textbooks, 
manuscript exercise-books made by pupils, and pupils’ maps 
and drawings; and under certain conditions all kinds of writ- | 
ten materials, if the problem is to observe what people un- 
consciously reveal about themselves rather than to determine 
what they consciously or deliberately say about themselves. 

If one were studying correspondence during the days before 
Noah Webster, to determine variations in spelling, rather than 
the messages deliberately recorded in the letters in question, 
such sources would properly be classified as remains. On the 
other hand, if these same letters are analyzed to secure infor- 
mation concerning attitudes toward public education, they are 
considered as documents. Obviously a diploma in blank is a 
remain, but when filled in with the name of “John Jones” and 
with testimony concerning his character and scholastic attain- 
ments, it becomes a document. Textbooks and exercise-books 
are not deliberately or consciously recorded descriptions of 
school practice and procedure during a given period, despite 
the light they throw on such problems, and therefore must be 
considered remains. However, the annual report or written ac- 
count by a person such as Horace Mann, or the oral testimony 
of an eye-witness, who observed these books in use and even 
quoted sections from them would be a document. 

In the case of a document, the brain of one human being 
and the attempt of the observer to record his impression of 
what happened have come between the original event and the 
user of the source materials—for example, the opinion of a 
judge, minutes prepared by the secretary of a board of educa- 
tion, a superintendent’s annual report, a school-survey report, 
a college catalogue, or a course of study. Thus, historical docu- 
ments, when used to learn what has been deliberately recorded 
in them, include everything that has been written about the 
past. The previously mentioned article by Professor Good con- 
tains a helpful classification of documents, prepared for workers 
in the history of education," 


17 Ibid.. D. 12. 
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Legislative acts such as constitutions, laws, charters 


. Court decisions 
III 


Executive and other official records 

A. Proceedings of administrative officers and bodies 

1. Minutes of boards of education 

2. Reports and orders of principals, superintendents, presi- 
dents 

3. Reports of committees including recommendations for 
executive action 

4. Systems of student records, salary lists, etc. 

Proceedings of deliberative bodies, such as the National 

Education Association, the North Central Association, etc. 

Reports of commissions; for example, “School Inquiry” 

Reports of school surveys and of official observers; for 

example, the report of Cousin 

Courses of study 

. Catalogues, prospectuses, advertisements 

Newspapers and periodicals 

A. Articles 

B. News notices 

C. Advertisements 

Personal materials 

Autobiographies, memoirs, reminiscences and biographies 

Annals and histories written by actors in the events narrated 

Letters 

Legal instruments executed by individuals in a personal 

capacity, contracts, wills, and deeds 

Legal instruments conferring powers upon individuals; for 

example, certificates 

F. Lecture notes 

Literary materials, as the novels of Charles Dickens or Edward 

Eggleston (All literature has a potential use in the history of 

education. A great amount of such material is found in the pub- 

lications of the Early English Text Society and similar bodies.) 


me oA w 


m Dow> 


Primary and secondary sources. In attempting to solve a 
definite, answerable, and significant problem in the history 
of education, it must be remembered that primary sources are 
the only solid bases of historical work. Primary sources are 
the original documents or remains, the first witnesses to a fact. 
Reference to the preceding lists of documents and remains 
provides illustrations of primary sources. The court stenog- 
rapher’s record and the newspaper reporter’s account of a trial 
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are primary sources (records of what the writers saw and 
heard), while an account of the trial (an editorial for exam- 
ple), by one not present, even though based on a stenographic 
report, tends to lose its primary character* Xn other words, 
only one mind, that of the observer of the event, should come 
between the original happening and the user of the sources. 

However, in one instance a source may be primary and in 
another secondary. Ordinarily textbooks in the history of 
education are secondary sources, usually many times removed 
from the original event, but for the worker who wishes to 
study the organization used by writers in this field these books 
become primary. 

It is necessary in some historical studies to begin with sec- 
ondary sources and to work back to the primary sources when 
the latter are not known in the beginning. For example, in 
making up a list of old arithmetic textbooks, it may be neces- 
sary to begin with encyclopedias, available articles and bibliog- 
raphies on the subject, histories of arithmetic, etc., which are 
secondary sources, and from these work back to the arithmetic 
texts, the primary sources, in this instance.'® 

In spite of insistence on the use of primary sources, members 
of graduate departments of instruction are able to cite ex- 
treme instances of apparently immature graduate students who 
seem totally unable or unwilling to go beyond a few secondary 
sources On a given educational topic. In a purported study 
of the educational history of a junior high school in a large 
city, the local newspapers and the reports of the school board 
were scarcely touched, the former being used only twice. In 
treating the development of art education in the same school 
system, a candidate for the Master’s degree devoted 122 of a 
tota] of 188 citations to five secondary sources. A student, who 
had previously mishandled a study involving a battery of 
standard tests, in turning to the educational development of 
a State school system over a given period of time, never seemed 


18 Jha., p. 43- 
19 Jbid., pp. 43-7, 74-6. 
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to get beyond a state history or two (secondary sources), ex- 
cept for the reminiscences of a minister and the notes of a 
federal officer. An historical investigation, in its preliminary 
form, of higher education in another state utilized almost en- 
tirely existing state histories, seemingly unaware of the mate- 
rials to be found in college catalogues, presidential reports, 
and other official records. As a matter of fact, the bibliography 
included only three college catalogues, one volume of statistics, 
two handbooks, one letter, and one annual state report. 

Bibliographical and library procedure. It will be recog- 
nized that historical research involves more intensive biblio- 
graphical work and library usage than either the experimental 
or survey types of research. The method of canvassing educa- 
tional literature and evidence was outlined rather fully in 
Chapter III. Form for bibliography and footnotes will be 
illustrated in Chapter XIII. Although the problem of docu- 
mentation is associated with the writing of the narrative, in 
the collection of data the beginning worker should be cau- 
tioned to assemble full bibliographical information in his note 
system to facilitate proper documentation. Valuable assistance 
in the compilation of the bibliography and in the use of avail- 
able library facilities may be secured from Hockett’s detailed 
outline,” and from Alexander’s briefer treatment.’ 

Note-taking and note systems. It is not desirable or neces- 
sary that the bibliography be completed before note-taking 
begins; the compilation of the bibliography, the study of 
subject-matter, and note-taking may well progress simultane- 
ously throughout the course of the investigation. Since excellent 
and detailed discussions of this phase of historical research 
are available,”? it seems adequate to present a brief summary 
statement of the procedures involved.** 


20 H. C, Hockett, op. cit., pp. 7-45. 

21 Carter Alexander, op. cit., pp. 212-5. 

22E. W. Dow, Principles of a Note-System jor Historical Studies. New York: The 
Century Co., 1924. Pp. 124. 

H. C. Hockett, op. cit., pp. 10-25 46-55. 

Carter Alexander, op. cit., pp. 80-90. 

22H. G. Good, “Historical Research in Education,” op. cit., pp. 77-8. 
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To make good progress in answering historical questions the student 
should collect the material systematically. For this purpose he will 
need a well-arranged plan of note-taking. W. H. Prescott because of his 
blindness deliberately acquired the ability to hold in mind and to 
place into an orderly sequence the details of long chapters of his 
works; but most people are not geniuses, If they want to have access 
to large quantities of detailed information they must write it down. 
This means a system of notes. Great historians like McMaster, James 
Ford Rhodes, and Edward Gibbon are known to have taken the notes 
for their extensive writings in bound note-books. Of this practice one 
can only say that they succeeded by means of a system that is not 
the best. 

A note-system should be flexible, that is, it should be possible to 
add to it at any point without disarranging the older material; and 
it should be possible to rearrange the notes at will. This requires that 
the notes be taken on separate sheets,:slips, or cards. Each piece 
should as far as possible contain a complete item, but when the matter 
to be noted is too extensive it may be continued on successive pieces 
and these numbered in series. Not more than one item should ever 
be placed upon a single piece. Each piece should have a subject- 
heading at the top and a margin for indexing, etc., at the left. For 
most kinds of historical note-taking two sizes of paper are desirable. 
What the sizes shall be will be determined by convenience in using 
and convenience in filing. Probably the most useful sizes are the 
3 x 5 or 4 x 6 cards for small items and the ordinary lettersize paper 
for larger passages. 

About three different kinds of notes are regularly made by historical 
workers. The first is the bibliographical note which always contains 
the standard data, author, title, pages, place and date of publication, 
and other formal facts about a document. It should usually have in 
addition a brief analysis of the contents and some account indicating 
the uses and the defects of the document. The second kind is the 
subject note which contains one item of information about a particular 
topic, with the source whence it was obtained. The great body of 
notes collected by any student will usually come under this head. One 
caution may be given here. Do not copy out long passages in readily 
accessible works but rather abbreviate and summarize. A third kind 
may be called the method notes. In collecting such material one con- 
stantly comes across suggestions or thinks of ideas which seem useful 
in interpreting the facts. Such suggestions or interpretative ideas do 
not fit into either subject or bibliographical entries, but they must 
be noted or they will be forgotten. Finally, when any extensive body of 
information is collected, an index and a more or less elaborate system 
of cross references become useful. Each card or sheet will indeed 
have. at the top. a title which gives in a word or two the particular 
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contents of that piece. These titles themselves form a sort of index. 
But the cards are constantly being rearranged for special purposes. A 
separate index, modified from time to time, will help in the final 
organization and discussion of the materials. 


CRITICISM OF DATA *4 


After the appropriate records of the past have been located, 
the historian must learn to read them correctly as a basis 
for developing sound ideas of the past, which in turn may 
aid in intepreting present trends and possibly in predicting 
more accurately what may happen in the future. 


He gathers data industriously, of course, but also resourcefully, 
noting particularly any facts which might tend to controvert his pre- 
formed views, all unpleasant and unwelcome evidence, scrutinizing 
any discrepancies and disagreements, and keeping a sharp eye for 
circumstances previously unnoticed or neglected. He looks not only 
for facts but for clues. Facts must lead to ideas. He must respect 
the facts, but in themselves they are blind. He must draw conclusions, 
because he aims not at knowledge merely, but wisdom. 


The historian subjects his sources to external and internal 
criticism. External criticism is concerned with the genuineness 
of the document itself, whether it really is what it purports 
or seems to be and whether it reads true to the original. In- 
ternal criticism deals with the meaning and trustworthiness 
of statements that remain within the document after any spu- 


24The present writers are greatly indebted to the following authors in this brief 
summary of principles of historical criticism: 

F. M. Fling, The Writing of History: An Introduction to Historical Method. New 
Haven, Conn.: Yale University Press, 1920. Pp. 196. 

H. C. Hockett, Introduction to Research in American History. New York: The 


Macmillan Co., 1931. Pp. 168. 
Allen Johnson, The Historian and Historical Evidence. New York: Charles Scribner's 


Sons, 1926. Pp. 180. 
J. J. Jusserand and Others, The Writing of History. New York: Charles Scribner’s 


Sons, 1926, Pp. vii + 143. 
C. V. Langlois and C. Seignobos, Introduction to the Study of History. New York: 


Henry Holt and Co., 1898. Pp. xxvii + 350. pas 
W. E, Spahr and R. J. Swenson, Methods and Status of Scientific Research. New 


York: Harper and Bros., 1930. Pp. xii + 534- g 
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Henry Holt and Co., 1911. Pp. 350. i 
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rious or interpolated matter has been removed from the text. 
External criticism sometimes is called lower criticism, since 
it is a preliminary and preparatory step, providing the data 
to be used in the second phase known as internal or higher 
criticism. However, there is no sharp line of demarcation be- 
tween these two phases of historical criticism; both progress 
simultaneously in many instances, with a great deal of over- 
lapping. In fact, external criticism employs internal evidence 
from the document (through a study of its contents), and 
internal criticism uses external evidence concerning authorship, 
time and place of writing, etc. These terms, external and in- 
ternal, refer to purpose and not to method or procedure, or 
whether one looks within or without the document to accom- 
plish the desired purpose. Both external and internal criticism 
may examine the contents of the document. Any differentia- 
tion between the two is in terms of the statements to be 
proved rather than the means employed to establish such proof. 
External criticism deals with data relating to form and ap- 
pearance rather than meaning of contents, while internal criti- 
cism weighs the testimony of the document in relation to 
the truth, However, despite the fact that external and internal 
procedures may not be differentiated sharply in the thinking 
of the mature historical worker, it is helpful for the beginner 
to discuss these two phases of historical criticism separately. 

External criticism. The first test which the historian applies 
to a document or remain is that of genuineness, called external 
criticism. Specifically, the problems of external criticism in- 
volve the questions of authorship and textual criticism to de- 
termine all the conditions which may have influenced the 
production of the document (time, place, purpose, and circum- 
stances of composition), and what part of the document is true 
to the original. Such questions as the following involve prob- 
lems of external criticism, and are also useful in the process 
of internal criticism, that is, in determining the truth and 
value of the statements made in the document.: 


26H. G. Good, “Historical Research in Education,” op. cit., pp. 17-18. 
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£. Who was the author, not merely what was his name but what 
were his personality, character, position, and so forth? 
2, What were his general qualifications as a reporter—alertness, 


character, bias? 

3. What were his special qualifications and disqualifications as a 

reporter of the matters here treated? 

(a) How was he interested in the events related? 

(b) How was he situated for observation of the events? 

(c) Had he the necessary general and technical knowledge for 
learning and reporting the events? 

4. How soon after the events was the document written? For one 
purpose the century of composition may be sufficient; for another the 
very hour may be essential. 

s. How was the document written, from memory, after copsulta- 
tion with others, after checking the facts, or by combining earlier 
trial drafts? 

6. How is the document related to other documents? 

(a) Is it an original source; wholly or in part? 

(b) If the latter, what parts are original; what borrowed; 
whence? How credible are the borrowed materials? 

(c) How and how accurately is the borrowing done? 

(d) How is the borrowed material changed; how used? 


The question of authorship is involved in discovering which 
numbers of The Federalist were written respectively by Madi- 
son, Hamilton, and Jay. The study of Washington’s “Farewell 
Address” raises the problem of determining what contribution 
each of three men made to it, since Madison drafted a portion 
of it, embodying the President’s suggestions, and Hamilton 
later extended and completed the draft, incorporating addi- 
tional suggestions made by Washington. It is thought that 
some of the presidential messages of Andrew Johnson were 
written by George Bancroft.?7 

Although there may be no great temptation to perpetrate 
hoaxes, frauds, and forgeries in the field of education, such 
trickery has been rather common in the fields of history, lit- 
erature, art, relics, antiques, business documents, natural sci- 
ence, and exploration. Various motives operate in the case of 
deception: (1) the use of a well-known name may increase the 
sale and prestige of a manuscript; (2) the reputation of promi- 


27H. C. Hockett, op. cit., pp. 63-64. 
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nent persons, who are too busy or who lack the necessary 
training to write effectively, may be increased through em- 
ployment of “ghost” writers; and (3) pseudonyms may be 
used to mystify or to stimulate the curiosity of the public. 
Among the numerous types of invention or forgery which 
appear are: (1) proverbs, epigrams, or witty sayings of fa- 
mous people, who somehow always seem to have had at tongue’s 
end a brilliant phrase or retort at the right moment; (2) 
invented speeches placed in the mouths of famous personages 
by the older historians, after the fashion of novelists; (3) 
insertion of applause in Congressional addresses which never 
were delivered; (4) genealogies and family trees; (5) inter- 
polations or insertions for deceptive purposes by copyists or 
others; and (6) business documents, works of art, relics, and 
antiques. 

The problems of genuineness are so varied and intricate 
that frequently separate auxiliary sciences are employed by 
the historian.?* 


Epigraphy is the study of inscriptions and the art of deciphering 
them. Diplomatics is the science of charters and diplomas and in- 
cludes a knowledge of the practices of chanceries and of the forms 
used in them. Paleography is the study of writing, which has a history 
all its own. The handwriting of a given scriptorium is usually quite 
characteristic. Besides, handwriting varies from age to age so that it 
alone is often quite competent to locate a manuscript in time. Philol- 
ogy in all its branches is of the greatest use in determining date 
and authenticity. To give some very elementary examples, the word 
“choose” was in the eighteenth century frequently spelled “chuse”; 
“clothes” was spelled “cloathes”; “entire,” “intire”; and so on. The 
young poet, Chatterton, it seems, first wrote out his verses in the 
English of his own day; but wishing to represent them as a “his- 
torical find,” he then proceeded to give them an antique air by 
substituting ancient words for modern ones. Not knowing the older 
language at first hand he resorted to Kersey’s dictionary for these 
older words. Kersey, however, had made numerous mistakes, and 
when Chatterton ignorantly copied these, he betrayed both the source 


28 H. G. Good, “Historical Research in Education,” op. cit, pp. 13-4. 
Also see J. M. Vincent, “History and Auxiliary Sciences,” in A Guide to Historical 
Literatsre, pp. 1-45. New York: The Macmillan Co., r931. 
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of his archaic vocabulary and the fact of his forgery.*° Archaeology, 
anthropology, and prehistory in general have had the greatest influence 
upon the study of history; and the first of these might almost be 
called the science of remains. Chemistry and the paper-maker’s art 
may be able to say, and have often said, that a given document, 
written on woodpulp, for example, and with a particular ink, cannot 
be older than the definite date when these materials were first manu- 
factured. Coins and medals are often of great historical value. Indeed, 
there is no field or item of human knowledge that may not be called 
upon to give evidence for or against the genuineness of some particular 
document or remain. 


Certain letters attributed to Lincoln contain errors of which 
he could hardly have been guilty.*° In a letter dated May 9, 
1834, is the phrase, “that North East quarter of Section 40.” 
Since Lincoln was an experienced surveyor, he must have 
known full well that a Congressional township contained only 
thirty-six sections, and could not have made such a blunder. 
The same letter states that a family named Bixby was “leaving 
this week for Kansas,” when the territory of Kansas was not 
organized and opened for settlement until 1854; this name 
in all probability was not even in use as early as 1834. 

It is true that since the invention and use of printing, as 
well as photography, in the reproduction and transmission of 
documents, the opportunities for error are greatly decreased. 
Before the time of these two inventions, when manuscripts 
and books were copied by hand, it was relatively easy for 
errors of omission and commission to creep into the author’s 
original work, which in the course of the centuries passed 
through the hands of many different copyists. 

Internal criticism. After questions of authorship and genu- 
ineness have been answered, and the actual language or text 
of the original document has been determined as nearly as 
possible, it remains for internal or higher criticism to determine 
the accuracy and value of the statements made. The shift of 


29R. L. Marshall, The Historical Criticism of Documents, p. 26. London: Society for 
Promoting Christian Knowledge, 1920. 

30 H. C. Hockett, op. cit., p. 66. 

P. M. Angle, “The Minor Collection: A Criticism,” Atlantic Monthly, CXLIII 
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emphasis is from the document as such to statements within 
the document. Obviously, proof of genuineness of a document 
by external criticism does not guarantee that it tells the truth. 
Although much of internal criticism is textual criticism, it 
also involves such factors as the competence, good faith, posi- 
tion, and bias of the author. Internal criticism is positive in 
nature when it seeks to discover the literal meaning and the 
real meaning of the text. It is negative when every possible 
reason is sought for disbelieving the statements made, ques- 
tioning critically the good faith and accuracy of the author. 
Of course, both positive and negative criticism are essential in 
historical research, although the student should not go so far 
as to be cynical and hyper-critical. 

The literal meaning and the real meaning are usually the 
same in modern documents, although such is not the case for 
many of the older sources. Actually to determine even the lit- 
eral meaning of Chaucer, Spenser, and Shakespeare is no small 
task for the young student, due to the presence of unfamiliar 
or obsolete terms and reference to strange institutions and 
customs. The student of history must know the language of 
the author and of the time and locality in which he wrote. 
The example of Chatterton’s poetry shows how language 
changes with time. The Jewish child who recited the line from 
Hiawatha, “Rose the firs with cones upon them,” as “Rose 
the Cohens with furs upon them,” simply did not know the 
language of Longfellow and of the forest. Even in modern 
documents the real meaning may not be clear due to allegory, 
symbolism, irony, satire, jests, hoaxes, allusions, implications, 
metaphors, hyperboles, and other rhetorical figures and lit- 
erary artifices. Certain political speeches and platforms are 
notorious examples of documents containing ambiguous mean- 
ings, intended as a rule to catch votes. 

Criticism directed toward questioning of the good faith, 
competence, and accuracy of the author is considered negative 
in character. It is reported that the deception of the man who 
pretended to be in touch at sea with the kidnappers of the 
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Lindbergh child was discovered partly through a too accurate, 
too detailed, neatly typed record covering the purported con- 
tacts with the kidnappers. Ordinarily one does not keep such 
a record under conditions of stress and strain involving loss 
of sleep, meetings at night on the ocean, nervous tension, etc. 
In other words, a document may be too perfect. 

The author’s motive for writing the document should be 
examined. Too profuse protestations of impartiality may be 
looked upon with suspicion, since, as Shakespeare had Ham- 
let’s mother say, “The lady doth protest too much, me- 
thinks.” * Accident or circumstance sometimes forces the 
writer to conceal the truth or to report only a half-truth; 
the relation between military reports and propaganda is well, 
known; lists of persons who are to be present at social func- 
tions are frequently in type before the hour of the event, 
without any later check on actual attendance. 

Distortion of fact may also result from personal vanity, a 
desire to flatter one’s superiors, bias, an attempt to please the 
public, or from literary artifice. John Smith’s vanity is assigned 
by critics as the reason for doubting the story of his rescue 
by Pocahontas, since it was not until after she had become a 
celebrity (and after her death) that Smith published his state- 
ment.?? The work of historians, artists, and poets subsidized 
by kings must be examined with great care, if the actual facts 
are to be sifted from the flattery which may be included. 
Desire to assign to an already great character the traits of 
extraordinary nobility may explain the Parson Weems story 
of “George Washington and the Cherry Tree.” * 

Bias is associated with such factors as religion, politics, 
class, exaggerated patriotism, and racial antipathy. Certain 
current books in psychology and biology reveal the influence 
of very definite religious dogmas. Histories have varied re- 
markably with the political, racial, national, or regional af- 
filiations of the authors—for example, the treatment accorded 

81 Hamlet, Act III, Scene 2, line 240. 
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the various wars in which the United States has participated, 
especially the Civil War and Revolutionary War. The utter- 
ances of certain contemporaries on public education and the 
curriculum must be evaluated in terms of the social and eco- 
nomic status represented and the vested interests involved. 
Frequently sentiments are expressed in political speeches and 
ceremonial addresses to please the public without strict ad- 
herence to the actual truth. 

When more than one statement concerning a given event 
is available, those which survive the processes of external 
and internal criticism are subjected to additional tests in 
terms of whether they are: (1) independent observations, (2) 
made by different workers, (3) belonging to different groups 
with varying affiliations, (4) operating under different con- 
ditions.** In general, statements made by alleged eye-witnesses 
are discredited, if opposed to the known body of science and 
of historical knowledge. Psychology provides sufficient ex- 
planation of the way in which so-called miracles are reported 
due to ignorance, illusion, hallucination, or extreme stimula- 
tion through the emotions or drugs. 


THE WRITING OF HISTORY 


After the data have been collected and criticized, there re- 
mains what is in many respects a much more difficult prob- 
lem, namely the synthetic and constructive process, or what 
may be called historical composition. 


Having collected some information upon a subject the student will 
desire to arrange it and present it in such form that others may get 
the benefit of his studies. This is partly a mechanical problem, the 
problem of documentation. It is partly a logical problem also, because 
it involves the question of the relative importance of the several 
items and topics. Finally, it is a philosophical and an artistic problem, 
because every historian, deliberately or in spite of himself, interprets 
what he presents. 


34 Ibid., p. 101. 
35H. G, Good, “Historical Research in Education,” op. cit, p. 78. 


THE HISTORICAL METHOD 265 


The mechanical problem of documentation is discussed in 
Chapter XIII. Other examples of forms for footnotes and bib- 
liography, especially adapted to the field of history, are avail- 
able,** and need not be reproduced here. 

Although the classification and analysis of data are treated 
in Chapter XI, brief comment may be made at this time, 
with special application to historical data. Earlier practice in 
historical writing advocated a strictly chronological type of 
organization involving a series of events broken up into short 
time-anits, with virtually no recognition of the possibilities 
of the topical or thematic grouping of materials. An appropri- 
ate combination of the chronological and topical organization 
of historical data seems best, involving consideration of such 
influences or forces as political institutions, law, economics, 
geography, natural resources, social conditions, war, national 
culture, race, education, art, literature, religion, great leaders, 
etc. A good example of the topical or functional organization 
of the history of education, combined with a definite time 
sequence, is cited earlier in this chapter." 

The successful writer must know the historical value and 
significance of each topic; in other words, he must possess 
historical perspective. Though many data, painstakingly gath- 
ered, may have to be discarded to promote condensation, con- 
centration, and precision, there should be left no opportunity 
for too many details to crowd out the main actors or events 
of the historical narrative. Too many of the older historians 
have departed from the main theme on side excursions which 
involved anecdotes or the subjective impressions of the given 
author. Herodotus in telling the story of Greece often in- 
serted an anecdote concerning some other country, which 
probably led to another similar tale, and by the time the 
author had returned to Greece, the reader had taken an 
imaginary trip to Babylon or Africa.°* 

Problems of interpretation such as formulation of hy- 


36H. C. Hockett, op. cit., pp. 116-30. 
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potheses, constructive reasoning, inference, use of analogy, 
cause and effect, generalization, and prediction ate not pe- 
culiar to history, and Chapters IV, XI, and XII are con- 
cerned with such topics. Concerning one of the most important 
and difficult of these problems, causation, early history dif- 
fered materially from modern history in that trivial causes 
were associated with important happenings and great social 
events? Thus, the rebuilding of Jerusalem was attributed to 
a dream of Darius, king of Persia. The cackling of a flock 
of geese saved Rome; and Cleopatra at the height of her ca- 
reer ended her life by the bite of an asp, leaving Rome free 
to develop further its great power. Therefore, the older historian 
would have it that the world was profoundly influenced by 
the cackling of geese and the bite of an asp. Even recent writ- 
ers have attributed great events or movements to divine 
providence, political causation, or some “great man.” Obyvi- 
ously, any adequate attempt to write the history of a people, 
which includes their institutions, must consider geography and 
physiography, the physical and mental characteristics of the 
people, cultural conditions (language, literature, customs, re- 
ligion, art, commerce, science, etc.), and political or govern- 
mental institutions (constitutional provisions, legislation, and 
court decisions).‘° However, psycho-analysis of the minds and 
motives of actors who played on the stage of history many 
years ago must be attempted, if at all, with great caution. 
The literary style of effective historians has conformed to 
the age in which they lived, for example, the work of Gibbon 
and Macaulay. Of course it is not expected that modern stu- 
dents will imitate the early American historians (Prescott, 
Motley, and Bancroft) who were patriots as well as literary 
artists in their fields; one, Bancroft, is even oratorical in his 
first editions.“ To-day, writing can be, and is, effective with 
greater simplicity, with less appeal to the emotions. However, 
it must be admitted that certain historical problems lend 
39 Ibid., pp. 263-4. 
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themselves with considerable difficulty to a graceful and inter- 
esting style, for example, the constitutional and legal basis of 
a state school system. Finally, certain reservations must be 
maintained in considering history as an art, in particular a lit- 
erary art. The details of the picture or of the novel are filled 
in through the sheer play of the painter’s or author’s imagina- 
tion. This, the historian will not do, if he is interested in the 
objectivity and detachment of science. For this reason certain 
gaps may be present in authentic history. 

Before undertaking research in educational history, the be 
ginning student should study carefully the available literature 
dealing with the problems of historical composition,“ prob- 
lems which have been presented here only in outline. In the 
chapter bibliography are numerous other discussions of the 
several problems of historical research. 


THE HISTORICAL METHOD APPLIED TO A SPECIFIC EDUCATIONAL 
PROBLEM (AN ILLUSTRATION) 


Now that the various aspects of historical method have 
been analyzed at some length, it should be profitable to the 
reader to cite an extended illustration of the manner in which 
this method is applied to a definite educational problem of 
current interest. To use a single problem, to which the various 
steps in historical research are applied, seems preferable to 
a different topic for each stage, since this represents a func- 
tional approach similar to work on a graduate thesis. For- 
tunately for present purposes, an admirable example is 
available.** 

42 F. M. Fling, op. cit, Chapters VII, VIII. 

H. C. Hockett, op. cit., pp. 112-41. 

Allen johnson, op. cit., Chapter VII. 

C. V. Langlois and C. Seignobos, op. cit., Book IIT. 

W. E. Spahr and R, J. Swenson, op. cit., Chapter XI. 

J. M. Vincent, op. cit., Chapters XXII-XXVI. 4 

43H. G. Good, “The Possibilities of Historical Research,” Journal of Educational 
Research, XXIX (October, 1935), 148-53. For a different illustration dealing with com- 
pilation of a list of arithmetic textbooks published within the present limits of the 
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Before we can examine the record intelligently we must come to it 
with a problem, What problem?...How did progressive education 
arise and develop? Any selection, and therefore this one, immediately 
involves us in a definition. What is progressive education? More than 
likely we do not exactly know; but we must begin and therefore we 
shall assume, tentatively and for the present, that progressive educa- 
tion is that complex of theories and practices which sets a child, that 
is an inquiring learner, in the midst of society and the world and 
says, “Whatever ye do, do for him”; which says that the school, if 
there is a school, exists for the child as he now is; which says that 
education is growth and life and that learning is involved in these; 
which demands for growth and life a rich and well-organized material 
and social environment in which teachers and children learn with and 
from each other by inquiring, discovering, making, sharing, and en- 
joying. That is progressive education for us, just now. Where did it 
come from? 

A little examination will show that there are at least four elements 
in this definition. There is the element of general educational theory. 
We might start with “the mother school” of Comenius. Many ideas also 
come proximately from Rousseau through Pestalozzi, who in turn 
furnished the starting-point for two great complementary writers, 
Herbart and Froebel. All these, and especially the last two, have been 
criticized and partially caught up in the current restatements of Dewey 
who is, however, strongly influenced by a pragmatic philosophy and 
by his interest in social institutions, especially industrial and demo- 
cratic institutions. In so far as progressive education is to be studied 
in the writers, we must go first to these and then to others more 
remote from the main line. Everywhere we must select the pertinent 
elements and reject those which do not belong to the tradition which 
we are studying. 

And there is the psychology of childhood which discloses a child 
who is not a blank of paper for teachers to write upon, as Locke 
had it, or the empty pitcher to be filled, of Dickens’ satire, but an 
organism with impulses, appetites, a thirst for knowledge and life, 
and purposes. Childhood implies capacity for growth; maturity, a 
corresponding social deficiency, for it often connotes an unwillingness 
to make new adjustments. When, not so long ago, the small child 
began to come into the school and his social and mental immaturity 
became apparent, the problem of education was presented in more 
extreme terms than ever before. Here again Comenius made a be- 
ginning; and perhaps the wisest of all Rousseau’s words was the 
phrase: “Begin by making a careful study of your children for it 
is clear that you know nothing about them.” It is equally clear that 
child study and genetic psychology are important elements in the 
development of progressive education. 
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In the third place progressive education demands a rich environ- 
ment of gardens, shops, school theaters, studios, printing presses, 
laboratories, and equipment of every kind. It is a fascinating task to 
trace how invention has made possible an educational environment 
which, as Dewey says, is not only for listening but also for doing. 
How society has furnished the materials to enrich the school and 
how the school has gone out into the world is all a part of the story. 

And there are the progressive schools and their practices. Perhaps 
we should begin with Vittorino who decorated the walls of “La Casa 
Giocosa,” the Joyful House, with frescoes of children at play and 
who played with his children, The infant school movement of the 
last century, Keilhau, whose famous pupil, Georg Ebers, has left 
an invaluable record of Froebel’s first school, the Kindergarten, the 
English Hazelwood of Thomas Wright Hill and his famous sons, must 
be considered. The Abbotsholme of Cecil Reddie was founded in 
1889. Its lineage can be clearly traced; and its descendants are 
Bedales, the Landerziehungsheime of Germany and the Ecole des 
Roches of France, all of which, if not modelled on Abbotsholme, were 
certainly inspired by it. Dewey’s small school in Chicago (1896), 
the Francis W. Parker School in the same city (1901), Meriam’s 
Elementary School at Columbia, Missouri (1904) are American ex- 
amples; but Felix Adler’s Workingmen’s School in New York (1880) 
and the resulting Ethical Culture School had many progressive features 
from the beginning. 

These four elements must certainly be studied if we wish to know 
how progressive education has come to be. Nor should we omit the 
influence upon education of science, of the humanitarian impulse, 
of democracy as an ethical ideal, of industry, and the influence 
upon the education of normal children of the study and education of 
mental and sensory defectives. All have made contributions to pro- 
gressive education. Beyond these contributing factors there is an 
extensive criticism of progressive education which is a part of the 
movement and the record. 

Some of the sources for a study of the development of progressive 
education are evidently accessible in books and magazines, That much 
has been indicated by the titles and names already given. Use the 
Readers’ Guide, Education Index, biographical dictionaries, general 
reference works, and files of the magazine, Progressive Education. 
Use the most general sources first, even textbooks, summary as they 
are, to get leads; but turn away from them as rapidly as possible to 
special and original sources. Make your bibliography as you go on, 
including both titles you have examined and those you will examine 
later. Much can be done by wise browsing in likely areas in a 
university library. That is how we secure the previously neglected 
and unnoticed facts. Recent institutional developments and current 
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practices may be obtained from those actively participating in the 
movement. 

It is not the best plan in most subjects to attempt a large, certainly 
not an exhaustive, bibliography at the start. We should begin with 
a few sources, take notes from them, providing carefully for adequate 
documentation, and gather new sources as we proceed. The notes 
should contain at least three kinds of materials: information on the 
subject with the references, leads for further information, and ideas, 
the things suggested by your investigation. Do not trust to memory. 
Nobody has a good memory in historical investigation. 

Having found some sources and gathered some materials, how shall 
we read and evaluate them? That demands two kinds of qualifications, 
technical and philosophical. By technical qualifications we mean ability 
to judge the accuracy of a report from internal and external evidence, 
ability to detect bias and to criticize it, to find corrective and amplify- 
ing sources. Every teacher knows how difficult it is to develop sound 
critical sense in students. How they love to repeat what they have 
read or have been told, just as they have read and heard it. There is, 
of course, not only credulity but also occasional hypercriticism. Both 
are to be avoided. Sound criticism demands open-mindedness, intel- 
ligence, information, and patient thought and investigation. By philo- 
sophical qualifications is meant that every investigator must have a 
position of his own, a considered view of education on the basis of 
which he judges. In progressive education there are several varieties: 
the lunatic fringe which showed itself, to take a distant example for 
safety’s sake, in Hamburg, Germany, after the World War; the merely 
diplomatic, half-unwilling kind; and the enthusiastic but critical and 
well-informed variety. Every historical writer on this or any other 
subject must interpret; and his interpretation will have more value 
if he has a conscious and deliberately formulated standpoint and 
philosophy. 


LEGAL RESEARCH IN EDUCATION 


Study of legal materials pertaining to educational problems 
is clearly a type of historical research, as illustrated in the 
list of doctoral dissertations and in the outline of source ma- 
terials presented earlier in this chapter. The same problems of 
collecting data, of criticism, and of interpretation, are present 
in legal research, and since these topics have been discussed 
earlier in this chapter they do not require emphasis here. 

Although the concern of the school administrator and board 
member with school law is apparent. the interest of the class- 
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room teacher in such problems can be justified without much 
difficulty. The right of the teacher to control children going 
to and from school, the teacher’s liability for injury to chil- 
dren while at school, and the court’s influence on the cur- 
riculum are all legal problems. There seems no logical reason 
why teachers and field workers should feel limited in their 
investigations to studies of present-day teaching methodology 
and curriculum reorganization. Much worth-while work can, 
and should, be done in the field of historical and legal research 
in education by way of providing a sound foundation for the 
evaluation and testing of current materials, methods, and types 
of school organization. It is true that the worker in legal 
research must have access to a law library, and this necessity 
has limited most investigations in this field to students on a 
university campus working toward graduate degrees. 

For purposes of research in education, the law may be classi- 
fied into: (x) statutory law (constitutional provisions and 
legislative or statutory enactments), and (2) case or common 
law (principles applied by the courts in deciding issues not 
covered by statutory law). Edwards urges the desirability of 
distinguishing between these two classes of law in order to 
prevent erroneous conclusions of a serious nature. 

Suppose that the constitution or the statutes of a state provide 
that school districts shall be liable for accidents to pupils while under 
the jurisdiction of the school authorities. A case comes into the court 
involving the liability of a school district for injuries sustained by a 
pupil while attending school. The court has no choice in the matter; 
it merely applies a clearly expressed rule of the statutes and holds 
the district liable. But suppose there is nothing in the statutes gov- 
erning the case. Then the court bases its decision on what it deems 
to be sound principles of public policy. The principle adopted by the 
court constitutes a common-law principle for that jurisdiction, 

A number of studies have been made of legislative enact- 
ments and constitutional provisions for the school, and of the 
history of educational legislation, but the organization of case 
law (decisions of the courts) concerning school issues has been 


44 Newton Edwards, “Methods and Materials of Legal Research,” Review of Educo 
tional Research, IV (February, 1934), 85-91. 
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rather neglected until the past few years. Much of the law in 
which school people are interested is found in court decisions. 

Edwards cautions the student of school law concerning prin- 
ciples to be observed in the evaluation of a case: ** 


In determining the authority of a case, one must keep clearly in 
mind the controlling facts. When the facts in different cases are 
similar in some respects but unlike in others, it is important to make 
certain that the differences are not of such a character as to distin- 
guish the one case from the other. Those who have had little experi- 
ence in evaluating court decisions are very likely to regard cases as 
identical when in reality they should be distinguished. For example, 
suppose that one is interested in the law governing the right of a 
school board to require vaccination as a condition of school attend- 
ance. A case is found in which a school board, on the outbreak of an 
epidemic of smallpox im the community, passed a tule excluding from 
school all children who had not been vaccinated. In another case, a 
school board made the same rule, but there was no smallpox in 
the school district. The two cases are not identical, for the facts are 
materially different. The existence of smallpox in the community is 
the distinguishing element. Before a case can be considered as an 
authority for another state of facts, the facts in the two situations 
must be sufficiently similar to warrant the application of the same 
legal principles. 

It is more difficult, however, to identify than to distinguish cases. 
Facts may be widely different and yet be governed by the same prin- 
ciples of law. One should seek, therefore, not so much to establish 
identity of fact as to establish identity of principle. As an illustration, 
a school board may wish to know whether it has authority to permit 
the use of a school-house as a social center. The fundamental issue is: 
May school property be used for purposes other than conducting a 
public school? All cases, therefore, having to do with such uses of 
school property should be examined. Again, a pupil publishes in the 
local newspaper a poem which is a “take-off” on the faculty. He is 
suspended until he apologizes. In another case a pupil is suspended 
because he attends a moving-picture show in violation of a school- 
board regulation. The facts in the two cases may seem widely different, 
but they have one important element in common: The school board in 
each instance is attempting to control the acts of pupils off the school 
grounds after school hours. Success in interpreting judicial decisions 
depends very largely on ability to distinguish cases in which the facts 
are very similar and to relate cases in which the facts seem to have 
little in common. 


45 Newton Edwards, “Where and How to Find the Law Relating to Public-School 
Administration,” Elementary School Journal, XXVII (September, 1926), 14-24. 
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Care, moreover, must be exercised in distinguishing that portion of 
a decision. which is authoritative and that which is only dicta. Not 
infrequently, a judge, in rendering an opinion, makes statements with 
regard to matters not before the court for settlement. These remarks 
may be merely illustrative. For example, if a court is reviewing a 
case testing the authority of a school board to prohibit membership in 
high-school fraternities and the statement is made that a school board 
would have the authority to exclude unvaccinated children, such a 
statement is only dicta and can have little, if any, authority. Only 
those portions of an opinion which deal with the issues before the 
court are authoritative. 


A helpful outline of primary sources of the law and of 
guides to primary sources is as follows: ** 


PRIMARY SOURCES OF THE LAW 


I. Constitutions 
1. Constitution of the United States 
A. Found in United States Revised Statutes, Federal Stat- 
utes Annotated, United States Compiled Statutes, and 
United States Statutes Annotated 
B. Found also in revised statutes, annotated statutes, com- 
piled statutes, or codes of the various states 
2, State constitutions 
A. Found in revised statutes, annotated statutes, compiled 
statutes, or codes of the various states 
II. Acts of Congress and state legislatures 
1. Acts of Congress 
A. Acts recently passed (advance edition) 
(1) Official 
(a) Slip laws issued by Superintendent of Docu- 
ments, Government Printing Office, Washington, 
D. C 
(b) Statutes of the United States, published at the 
end of each session of Congress, commonly 
known as “Pamphlet Laws” 
(2) Unofficial 
(a) Federal Reporter Supplement 
(b) Federal Statutes Annotated Supplement—issued 
quarterly in pamphlet form 
(c) United States Compiled Statutes Annotated 
(i) Pamphlet supplements issued at short in- 
tervals 


46 Ibid., pp. 15-20. Also see Carter Alexander, op. cit., pp. 200-9. 


274 EDUCATIONAL RESEARCH 


B. Acts in force 
(1) Official 
(a) United States Revised Statutes, 2d ed., 1878. 
Contains statutes in force December 1, 1873 
(i) Supplement, Vol. 1, 1874-91 
(ii) Supplement, Vol. 2, 1892-1901 
(b) United States Statutes at Large, 1901 to date 
(2) Unofficial 
(a) Federal Statutes Annotated, 2d ed. Contains all 
general laws of the United States in force 
January 1, 1916 
(i) Annual supplements, 1918 to date 
(ii) Quarterly pamphlet supplements 
(b) United States Compiled Statutes Annotated, 
1916 
(i) Annual supplements 
(ii) Pamphlet supplements issued at short in- 
tervals 
(iii) Advance sheets published in Federal Re- 
porter 
2, Acts of state legislatures 
A. Session laws. At the end of each session of the legis- 
lature each state publishes in one or more volumes all the 
laws passed at that session. These are commonly known 
as “session laws.” 
B. Revisions. Called: Compiled Statutes, Consolidated 
Statutes, Revised Statutes, or Codes. 
IJI. Court decisions 
1. United States Supreme Court Reports 
A. Official 
(1) United States Reports. (These reports are cited as 
follows: x Dall. 56, 10 Cranch 67, 92 U.S. 98.) The 
first ninety volumes are cited by the name of the 
reporter. 
(a) Dallas (1-4 U.S.) 
(b) Cranch (5-13 U.S.) 
(c) Wheaton (14-25 U.S.) 
{d) Peters (26-41 U.S.) 
(e) Howard (42-65 U.S.) 
(f) Black (66-67 U.S.) 
(g) Wallace (68-90 U.S.) 
(h) United States, 91 to date 
B. Unofficial 
(1) Supreme Court Reporter. (Cited as follows: 185 S. 
Ct. 45.) Contains cases from 1882 to date. Current 
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opinions may be found in advance sheets issued dur- 
ing term time of court. 

(2) United States Supreme Court Reports, Lawyers’ 
Edition. (Cited as follows: 24 L. Ed. 231.) Contains 
all opinions of Supreme Court to date, Current opin- 
ions may be found in advance sheets issued during 
term time of court, 


2, State court reports 
A. Official 
(1) State reports. Each state publishes reports of its 
courts of final jurisdiction. (Cited as follows: 142 
Ill. 75, 95 Ga. 68.) 
B. Unofficial 
(1) National Reporter System. Decisions of all courts of 
authority, state and federal. (Cited as follows: 18 
Atl. 58, 24 N.E. 70, 10 Pac. 100.) 
(a) Atlantic Reporter, 1885 to date: Connecticut, 
Delaware, Maine, Maryland, New Hampshire, 
New Jersey, Pennsylvania, Rhode Island, Ver- 
mont 
(b) Northeastern Reporter, 1885 to date: Illinois, 
Indiana, Massachusetts, New York, Ohio 
(c) Northwestern Reporter, 1879 to date: Dakota 
Territory, Iowa, Michigan, Minnesota, Nebraska, 
North Dakota, South Dakota, Wisconsin 
(d) Pacific Reporter, 1883 to date: Arizona, Cali- 
fornia, Colorado, Idaho, Kansas, Montana, 
Nevada, New Mexico, Oklahoma, Oregon, Utah, 
Washington, Wyoming 
(e) Southeastern Reporter, 1887 to date: Georgia, 
North Carolina, South Carolina, Virginia, West 
Virginia 
(f) Southern Reporter, 18837 to date: Alabama, 
Florida, Louisiana, Mississippi 
(g) Southwestern Reporter, 1886 to date: Arkan- 
sas, Indian Territory, Kentucky, Missouri, Ten- 
nessee, Texas 
(2) Selected cases 
(a) Lawyers’ Reports Annotated, 1888-1905. (Cited 
as follows: 10 L. R. A. 57.) 
(i) Lawyers’ Report Annotated, New Series, 
1906-14. (Cited as follows: 42 LR A: 
(N.S ) oo.) 
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I. Digests 
American Digest System, 1658 to date: a digest of all de- 
cisions of American courts of last resort 


I. 


(2) 


(0) 


(c) 


(a) 


(e) 


(f) 
(g) 
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(ii) Lawyers’ Report Annotated, 1915A, 19164, 
etc., to 191E. (Cited as follows: L. R. A. 
IQI5À 21.) 

(b) American and English Annotated Cases, 1909- 
16. Continued by Annotated Cases to 1018 
(Cited as follows: Ann. Cas. 1913E 1324.) 

(c) American Law Reports, 1919 to date. A con- 
tinuation of Lawyers’ Reports Annotated and 
Annotated Cases. (Cited as follows: 33 A. L. 
R. 59.) 

(d) American Decisions, 1760-1869. (Cited as fol- 
lows: Am. Dec. 45.) 

(e) American Reports, 1870-87. (Cited as follows: 
15 Am. Rep. go.) 

(f) American State Reports, 1882-1912. (Cited as 
follows: 20 Am. St. Rep. 56.) 


GUIDES TO PRIMARY SOURCES OF THE CASE LAW 


Century Edition, known as the Century Digest: a digest 
of all decisions of American courts of last resort, t658- 
1896, Volume 43, topic, “Schools and School Districts” 
Decennial Edition, known as First Decennial Digest: a 
digest of all decisions of American courts of last resort, 
1897-1906, Volume 17, topic, “Schools and School Dis- 
tricts” 

Second Decennial Edition, known as Second Decennial 
Digest: a digest of all decisions of American courts of 
last resort, 1907-16, Volume 20, topic, “Schools and 
School Districts” 

Third Decennial Edition, known as Third Decennial 
Digest: a digest of all decisions of American courts of 
last resort, 1916-26, Volume 24, topic, “Schools and 
School Districts” 

American Digest, Current Digest: a digest of all de- 
cisions of American courts of last resort, 1926 to date, 
topic, “Schools and School Districts” 

American Digest, Monthly Edition: advance sheet 
pamphlets giving digest of current decisions 

Descriptive Word Index to Decennial and all Key-num- 
ber Digests, with supplements to date 
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2. State digests (As a rule, it is better to rely on the American 
Digest System), digest of the decisions of the courts of au- 
thority of a particular state (Examples: Massachusetts Di- 
gest, Kansas Digest, Maryland Digest) 

II. Encyclopedias 

x, Cyclopedia of Law and Procedure (1912), Supplements to 
1921, Volume 35, extensive treatment of school law under 
topic, “Schools and School Districts” (Cited as follows: 35 
Cyc. 1042) 
Ruling Case Law, completed in 1920 with supplements to 
date, Volume 24, topic, “Schools,” a treatise on school law 
based on leading cases, extremely helpful (Cited as follows: 
24 R. C. L. 550) 

3. Corpus Juris, Volume 56, topic, “Schools and School Dis- 

tricts” 

JII. Indexes to annotated cases 
Index to Notes in Lawyers’ Reports Annotated, 1888-1918 
Index to Notes in Annotated Cases, 1908-1918 
American Law Reports, cumulative Index-Digest, 1919 to 
date 
TV. Citations 

1. Shepard’s Citations: separate volumes for the courts of the 
United States and for the courts of each state issued at fre- 
quent intervals; indicates where a case has been cited by 
other courts and enables one to trace the history of a case 
and to evaluate it as of the present date 

V. Indexes to legal periodicals 

1, Jones’ Index to Legal Periodicals, to 1899 

2. Chipman’s Index to Legal Periodical Literature, 1898-1908 

3. Index to Legal Periodicals, 1908 to date: Quarterly with 
annual cumulations 


» 


vna 


The student of school law will find Edwards’ detailed analy- 
sis of the legal basis of school organization and administration 
the most comprehensive treatment available, covering the 
topics of: the school and the state, district organization and 
control, school districts and municipalities, school officers, 
legal authority of boards of education, school-board procedure 
and records, contractual authority and liability of school 
boards, the school money, the school debt, acquisition and 
use of school property, the contractor’s bond, tort liability 
of school districts, personal liability of school officers, employ- 
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ment of teachers, dismissal of teachers, pensions and minimum 
wages, school attendance, rules and regulations of boards of 
education, and discipline and punishment of pupils. Two 
other useful surveys of similar legal problems should be men- 
tioned.*® 

A valuable bibliographical and summary review of the litera- 
ture in this field includes 398 references which cover the topics 
of: federal and state relations to education, the constitutional 
basis of public education, state administrative organization, 
district organization and control, administration of local school 
systems, school finance, school property, tort liability of 
school districts, officers, and teachers, the legal status of teach- 
ers, the legal status of pupils, textbooks and curriculum, and 
legal aspects of higher education.*® Suggestive discussions of 
a more general type, without specific reference to legal prob- 
lems in education, deal with jurisprudence as a field for re- 
search,°° the politico-juristic approach to’ problems in social 
science,” and the use of the law library.*? 

Numerous studies and theses of a legal character are listed 
annually under the headings of “educational laws and legis- 
lation” or “educational legislation” in the bibliography of edu- 
cational research issued by the Office of Education." The 
scope of the problems in this field may be illustrated by the 
following titles of Doctors’ dissertations: 


x. “Constitutional Limitations on Legislation for the Common 
School.” 


2. “A Study of the Control of Education through Freeholders 
Charters of California Cities and Counties.” 


47 Newton Edwards, The Courts and the Public Schools. Chicago: University of 
Chicago Press, 1933. Pp. xvi + 582. 

48H. R. Trusler, Essentials of School Law. Milwaukee, Wis.: Bruce Publishing Co., 
1927. Pp. xlvi + 478. 

Frederick Weltzin, The Legal Authority of the American Public School. Grand Forks, 
N. D.: Mid-West Book Concern, 1931. Pp. 286. 

49 “The Legal Basis of Education,” Review of Educational Research, III (December, 
1933), 369-468. 

50 Roscoe Pound, “Jurisprudence,” Research in the Social Sciences, Chapter VI. 
Edited by Wilson Gee. New York: The Macmillan Co., 1929. 

51H. W. Odum and Katherine Jocher, op. cit., Chapter XI. 

52 W. E. Spahr and R. J. Swenson, op. cit., Chapter IX. 

5a Ruth A, Grav. ab. cit., and other annual bibliographies of the Office of Education 
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3. “Education in Recent Constitutions and Concordats.” 

4. “The Influence of Court Decisions in Shaping School Policies 
in Mississippi.” 

s. “Legislation and Legal Status in the Tax Supported High 
Schools of the State of New York.” 

6. “Legal Basis of Educational Control: A Study of Centraliza- 
tion.” 

7. “The Administration of the Local Budget Law in Iowa.” 

8. “The Legal Liability of the School District for Damages.” 

9. “The Legal Implications of the Concept of Education as a 
Function of the State.” 

to. “The Constitutional and Legal Basis of Education in New 
Jersey.” 

11. “Legal Aspects in the Administration and Control of Public- 
School Property.” 

12. “Legal Rights and Obligations of Public-School Students.” 

13. “The Constitutional and Legal Basis of Public-School Ad- 
ministration in Virginia.” 

14. “The Constitutional and Legal Bases of Public-School Ad- 
ministration in Oklahoma.” 

15. “A Legal Study of the Centralization of State Control of Edu- 
cation in Indiana (as Revealed by Constitutional Provisions, Legisla- 
tive Enactments, and Supreme and Appellate Court Decisions).” 
[These legal studies of state school systems have become numerous. | 

16. “State Constitutional Provisions for Education: Fundamental 
Attitude of the American People Regarding Education as Revealed by 
State Constitutional Provisions.” 

17. “Legal Status of the County Superintendent.” 

18. “The Contractual Status of California City School Superin- 
tendents.” 

19. “The Legal Authority of the American Public School as De- 
veloped by a Study of Liability to Damages.” 


BIBLIOGRAPHICAL AND SUMMARIZING STUDIES 


A type of exhaustive bibliographical survey, and of critical 
analysis and interpretation of results, found in certain of the 
summaries of educational research in given fields, approaches 
in character the technique of historical investigation, especially 
where a time span of some length is involved. The same proc- 
esses of collection of data, criticism, and interpretative writing 
are involved. The first-hand revorts of the various investiga- 
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tions are the primary sources. In the process of criticism 
questions concerning the good faith, competence, accuracy, 
bias, and position of the author must be answered, and the 
study under consideration must be compared with other in- 
vestigations in the same field. The writing of the report, 
involving the interpretative, synthetic, and constructive proc- 
esses, requires no less skill and insight than biography, legal 
research, and other types of historical investigation. In many 
instances it should be possible through a comparison and syn- 
thesis of results from a number of related studies to reach 
sounder generalizations or to make more definite the con- 
clusions previously formulated in somewhat tentative fashion. 
A careful survey of the existing literature relating to a given 
problem, discussed in Chapter III, should observe the prin- 
ciples enumerated in the present chapter, dealing with collec- 
tion, criticism, and interpretation of data. 

In most graduate schools this type of work is not encouraged 
as a thesis problem, since it is not generally considered as 
adding new truth to the accumulated store of educational 
knowledge. Yet, viewed in another light, a critical, analytical 
synthesis and interpretation of studies in the field of reading 
from 1880 to 1925 does add definitely to knowledge by way 
of indicating major generalizations, trends, and needed research 
not clearly discernible as long as several hundred investiga- 
tions were viewed separately rather than in relationship to 
each other and to a pattern representing the entire field of 
research in reading. Possibly one reason for reluctance to en- 
courage this type of study as a thesis problem is the existence 
of many superficial so-called summaries of the running- 
bibliography or term-paper variety. However, there can be no 
question about the practical value of such contributions to 
educational literature, and as sufficient studies accumulate, ap- 
propriate summaries should be worked out in each of the 
subdivisions of education. It is true that the Review of Edu- 
cational Research does this work in part, although less com- 
pletely and analytically than in the case of available digests 
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of investigations in such fields as reading; ** arithmetic; *° 
science teaching; ™® language, grammar, and composition; ** 
and the appreciation aspects of English.** 

Concluding statement. This chapter has discussed the na- 
ture, value, scope, and frequency of historical studies in educa- 
tion. In connection with the collection of data, documents and 
remains have been illustrated and emphasized as the primary 
sources, the only solid bases for historical research. In this 
chapter and elsewhere, stress has been placed on appropriate 
library procedure and an adequate system of taking and or- 
ganizing notes. The processes of external and internal criticism 
of the data collected have been analyzed, at least in terms 
of the major principles involved. Problems pertaining to the 
actual writing of educational history have been outlined briefly. 
Attention has been directed in particular to the significance 
of the legal and summary types of investigation in education. 

In presenting the systematically organized discussion of 
this chapter and in differentiating between primary and sec- 
ondary sources, documents and relics, internal and external 
criticism, etc., the authors are aware that the trained historical 
worker pays little conscious attention to such classifications. 
The experienced educational historian has developed his 
imagination and insight with respect to sources and criticism 
to such an extent that any formal organization of the pro- 
cedures is unnecessary. Since such investigators are relatively 


54W. S. Gray, Summary of Investigations Relating to Reading. Supplementary Edu- 
cational Monographs, No. 28. Chicago: Department of Education, University of Chicago, 
1925. Pp. viii + 276. 

55G. T. Buswell and C., H. Judd, Summary of Educational Investigations Relating to 
Arithmetic, Supplementary Educational Monographs, No. 27. Chicago: Department of 
Education, University of Chicago, 1925. Pp. viii + 212. 

56F, D. Curtis, A Digest of Investigations in the Teaching of Science in the Ele- 
mentary and Secondary Schools, Philadelphia: P. Blakiston’s Son and Co., 1926. Pp. 
x + 342. 


, Second Digest of Investigations in the Teaching of Science. Philadelphia: 
P. Blakiston’s Son and Co., 1931. Pp. xx + 424. É 

57R. L. Lyman, Summary of Investigations Relating to Grammar, Language, and 
Composition. Supplementary Educational Monographs, No. 36. Chicago: Department of 
Education, University of Chicago, 1929. Pp. viii +302. _ 

58R, L. Lyman, The Enrichment of the English Curriculum. Supplementary Educa- 
tional Monographs, No. 39. Chicago: Department of Education, University of Chicago, 
1932. Pp. viii +252. 
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few, it has seemed wise to present the foregoing analysis of the 
principles of historical research, even at the risk of seeming 
academic and formal, and of representing the historical method 
as more complicated than it actually is. Obviously, it is not 
to be assumed that mere reading of this chapter will take the 
place of needed additional training which comes through actual 
participation in historical investigation. 

Not long ago the three hundredth anniversary of the estab- 
lishment of secondary schools in this country was celebrated 
in educational literature and at national professional meetings 
with appropriate historical resumés of various phases and 
trends in secondary education. During the years of the recent 
economic and social disturbance, students of social science 
have examined more carefully than for a long time the founda- 
tions of society’s institutions and the cycles of change through 
which organized society has passed, seeking to evaluate pres- 
ent conditions and to discern possible future trends. In the field 
of education there is every indication of returning to historical 
method as an ally of the scientific movement rather than to 
depend so largely for the solution of educational problems 
upon a questionnaire canvass of practice, a battery of stand- 
ardized tests, a survey program, majority opinion, or statistical 
calculations. 


PROBLEMS AND EXERCISES 


1. Select a completed historical study, and analyze it in terms of the 
principles of research developed in this chapter, for example: 
(a) A list of the sources used—documents and remains, primary 
and secondary 
(b) Illustrations of the type of criticism employed—internal and 
external 
(c) Type of writing and of historical interpretation represented, 
and effectiveness of organization 
(d) Adequacy of documentation and bibliography 
2, Choose an historical study which might well be made (if desired, 
some aspect of the way in which the instructional program kas been 
adapted to pupil or social needs), and outline it in terms of: 
fq) Initial library procedure 
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(b) Possible sources of data 
(c) Tentative sub-topics 
(d) Brief bibliography (See Chapter XIII for form) 
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CHAPTER VII 


THE NORMATIVE-SURVEY METHOD: SURVEY TESTING, 
QUESTIONNAIRE INQUIRIES, DOCUMENTARY 
FREQUENCY STUDIES 


Introduction. This chapter deals with the second major type 
of research discussed in this volume. It is concerned with 
ascertaining the conditions which prevail in a group of cases 
chosen for study, and is essentially a method of quantitative 
description of the general characteristics of the group. Because 
of the breadth of the subject and the necessity for describing 
data-gathering instruments, the treatment is given in two 
chapters. The present chapter deals with studies employing 
survey-testing, questionnaire, and documentary-frequency pro- 
cedures. Interview, observational, and survey-appraisal pro- 
cedures are treated in Chapter VIII. 

The nature and the common uses of each procedure are 
given, methods of preparing data-gathering schedules are dis- 
cussed, and certain cautions for interpretation of the findings 
are included. 


APPRAISAL OF THE NORMATIVE-SURVEY METHOD 


Much research has as its object the ascertaining of laws 
which govern relationships; but such is not the sole purpose 
of research. People are interested also in other kinds of knowl- 
edge. For example, the preceding chapter has dealt with meth- 
ods used by workers who wish to find out what used to be the 
situation in the world, or in some particular section of it, 
before the present generation came upon it. This historical 
interest is often generalized and may be extended down to 
ihe very recent past. By methods which are appropriate to 

286 


THE NORMATIVE-SURVEY METHOD 287 


such a field of inquiry, workers are enabled to piece together 
bits of evidence to prepare a description of conditions at an 
earlier time, or to form a story of the succession of events, 
and thus tell how one condition led to another. 

In the present chapter we are primarily concerned with his- 
tory in the making. Having become interested in the study of 
large social problems, people wish to make and record pic- 
tures of these problems as they are developing, instead of 
waiting till the events are long past and scholars must ferret 
their story out of musty records. We do not mean to suggest 
that history will not be written so as to cover the present 
period, as it has the past, but certainly historians will not find 
it as difficult to gather many of the facts for the present era. 
The main point, however, is that society is taking a more sci- 
entific interest in the control of its affairs. Having seen what 
lessons can be learned from studying past conditions and 
events, people now demand information about present activities 
while that information can still be used to guide them in 
formulating policies and working out plans for development 
in the immediate future. 

Normative-survey research is directed toward ascertaining 
the prevailing conditions. It seeks to answer the question, 
“What are the real facts with regard to the existing condi- 
tions?” For example, a superintendent may want to know 
how many of the teachers in his system have college degrees. 
How many hold life certificates? How do these figures com- 
pare with the tendency throughout the state, or the nation? 
How many minutes per week are normally devoted to the 
teaching of spelling? What is a reasonable cost for the educa- 
tion of a kindergarten child? What proportion of a school 
budget is it reasonable to set aside for night school, or for 
adult education? What kind of a curriculum do people really 
want their children to have? At what age and grade level do 
pupils leave school? What happens to children after they leave 
school—what occupations do they choose, or what higher in- 
stitutions do they enter? These will suggest the countless 
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questions that arise concerning current conditions in the edu- 
cational world. 

It will be apparent that this type of information is par- 
ticularly important for the administrator. What is more natu- 
ral than that one should want to know how other school sys- 
tems are being run, so that he can compare his practices with 
theirs? By this means he can take courage in noting that 
certain of his procedures and standards appear to be superior 
to those of certain other schools; he will also have his atten- 
tion called to the superiority of other systems in certain points 
to which he should give more attention; and perhaps, hope- 
fully, he will get suggestions as to how to meet some of his 
pressing problems. Even if he does not secure any helpful 
suggestions, it is at least comforting to learn that other admin- 
istrators are also grappling with problems similar to his own. 

Interest in general conditions and practices is not limited 
to the administrator, though it is perhaps more characteristic 
of his work than of the work of others. Teachers, however, 
are concerned with the conditions obtaining in their classrooms 
and give frequent tests of one kind or another to keep in- 
formed of these conditions. They will also read widely about 
methods of instruction which others are using, and they will 
visit other classrooms to see what practices are employed. Al- 
though such activities will not ordinarily be classed as research, 
they serve to illustrate the interests for which normative- 
survey research is appropriate. Information-getting activities 
may be regarded as research only when they are relatively 
formal and carefully planned, as was pointed out earlier, in 
the first chapter. 

It should not be thought that research of the normative- 
survey type is peculiar to education, or to the social sciences. 
It is, for example, a significant mode of attack in any field of 
knowledge where geographic distribution is involved, such as 
the ecology of flora and fauna. The climatic conditions of 
various parts cf the world at any particular time are the sub- 
ject of considerable investigation ard reporting. The distribu- 
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tion of natural resources, such as forests, minerals, coal, 
petroleum, natural gas, water power, etc., is the subject of 
extensive and important surveys. The normative-survey ap- 
proach is appropriate wherever the objects of any class vary 
among themselves and one is interested in knowing the extent 
to which different conditions obtain among these objects. 

The compound adjective “normative-survey” is applied to 
this method in order to suggest the two closely related aspects 
of this kind of study. The word “survey” indicates the gather- 
ing of data regarding current conditions. The word “norma- 
tive” is used because surveys are frequently made for the 
purpose of ascertaining what is the normal or typical condition, 
or practice. 

Historical perspective may be secured. This direction of 
attention toward existing conditions does not, however, mean 
that the significance of normative-survey research is limited to 
the present. A study of the status of conditions at any given 
time may be repeated later, thus affording descriptions at dif- 
ferent periods of time so that comparisons can be made, the 
direction of change noted and evaluated, and future growth 
guided. This is of no small importance in a society that is 
“as complex and rapidly changing as ours. And it hardly re- 
quires mention that such studies furnish an increasingly valu- 
able source of information for the historian, Therefore, al- 
though the survey approach may at first appear superficial, in 
that it gathers data only for the present and does not im- 
mediately give the perspective of a long historical sweep, such 
perspective is ultimately made possible through a series of 
cross-section pictures of conditions at different times. 

A study of current status should, wherever possible, make 
use of similar information secured by earlier studies, so as to 
interpret present findings in the light of changes and trends, 
For example, a recent study of the status of city school super- 
intendents + derived increased significance from the fact that 


1“The Progress and Status of the City Superintendent of Schools,” Chapter VI, 
pp. 99-149, in Educational Leadership. Eleventh Yearbook of the Department of Supet- 
intendence. Washington, D. C.: National Education Association, 1933. Pp. 528. 
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a similar study had been made ten years previously, and com: 
parisons could be drawn on many different points, such as 
amount of training, length of experience before becoming 
a superintendent, etc. The recent investigation therefore shows 
not only the present situation, but when compared with 
the earlier study, indicates the direction in which changes 
are taking place. A study of the holding power of public 
schools was made several years ago by Kline? which 
purpe ¿ly repeated as closely as possible an investigation 
made by Thorndike* about twenty-five years earlier. This 
duplication enabled Kline to report that the proportion of 
students staying in school long enough to enter high school 
had increased four times. When the workers in the National 
Survey of Secondary Education (1929-1931) wanted a definite 
test of whether the high schools of the country were becoming 
more democratic,’ they compared their findings with earlier 
studies by Counts ê and others. 

The fact that we do not have more instances of such per- 
spective in present normative studies is primarily because stu- 
dents in the past have not produced a large number of such 
investigations, It might fairly be said that only during the 
present generation has any widespread recognition of the 
importance of normative surveys developed. At the present 
time many people in all fields of social inquiry are recognizing 
that, by making numerous, carefully executed status studies, 
they are rendering a much needed service not only to the 
contemporary generation but to generations for many years to 
come. 3 


2The Status of the Superintendent. First Yearbook of the Department of Super- 
intendence. Washington: National Education Association, 1923. Pp. 206. 

3 Elias J. Kline, “Significant Changes in the Curve of Elimination since 1900,” 
Journal of Educational Research, XXVI (April, 1933), 608-16. 

4 Edward L. Thorndike, The Elimination of Pupils from School. U. S. Bureau of 
Education Bulletin, 1907, No. 4. Washington, 1907. Pp. 63. 

5 Grayson N. Kefauver, Victor H, Noll, and C. Elwood Drake, The Secondary-School 
Population, Chapter I, “Changes in the Secondary-School Population,” pp. 1-26. U. S. 
Office of Education Bulletin, 1932, No. 17; National Survey of Secondary Education 
Monograph Nu. 4. Washington, 1933. Pp. 58. 

6 George S. Counts, The Selective Character of American Secondary Education. Sup- 
plementary Educational Monographs, No. 19. Chicago: University of Chicago, 1933. 
Pp. 162. 
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Relation of normative-survey research to practical needs. 
One may be prompted to ask whether normative-survey data 
solve problems. To this it must be replied at once that prob- 
lems of a practical nature are not solved directly by data of 
any kind. The solving of problems is a distinctly psychological 
process. Solutions do not lie in data; they result from think- 
ing, with the help of the increased insight which grows out 
of a study of data. It is important to know the psychological 
side of research, and to distinguish between facts and infer- 
ences. As for the value of normative-survey data in affording 
a basis for inferences that may aid in solving practical prob- 


lems, it may be said that this kind of data will probably be} 


more highly regarded by the administrator in helping him 
solve practical problems than are the principles and laws grow- 
ing out of experimentation in the laboratory. The reason is 
this: the data coming direct from the field represent field con- 


ditions; they tend to be practical because they grow out offi 


practical situations; and they generally answer the questions 
of the man in the field because they are likely to be cast in 
the terms in which he thinks. 

There, is, of course, another aspect to the practical situa- 
tion: radical changes, or adaptations involving great shifts, are 
not commonly contemplated in practice. For this reason a theo- 
retical, scientific principle discovered by carefully controlled 
experimentation would not be immediately acceptable as a 
standard for practice if it required a very sharp departure from 
current usages. This condition is not a reflection on school ad- 
ministration; it is characteristic of administration in all fields 
and is simply an expresson of the fact that the administrator 
is expected to take many factors into consideration rather 
than to make great concessions to any one factor at the ex- 
pense of others. In the manufacture of automobiles, for ex- 
ample, the engineers keep years ahead in their inventions and 
designs; the board of directors studies carefully the problem 
of introducing these new developments so that their effect on 
Sales will be beneficial. In fact, a group of directors may be 
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more influenced in their decision by what other manufacturers 
are planning to do (normative data) than by the new ideas 
their engineers have recently evolved. Such practices empha- 
size the fact that running a large enterprise necessitates the 
consideration of many different things, of which technical con- 
siderations form only one group. These practices also suggest 
the importance of normative data in many walks of life. 

Bureaus of research in city school systems, in state depart- 
ments of education, in teachers organizations, and in other 
field situations are likely to give a predominant portion of 
their time to studies of the normative-survey type. This is in- 
dicated by several surveys of the work of field research bu- 
reaus, but it is stressed most pointedly in a bulletin of the 
National Survey of Secondary Education.” Perhaps the clear- 
est indication of this tendency is found in the kinds of statis- 
tical analysis which are employed by these bureaus. In one 
of his tables (p. 65) Zeigel shows that such measures as per 
cent, median, mean, range, rank, etc., are employed generally 
in field studies, whereas correlation is used in less than 5 per 
cent of the studies which were analyzed. The former measures 
are those which typify the survey type of investigation, and 
thus indicate that research bureaus devote the greater portion 
of their time to this type of study. This tendency of research 
bureaus throws further light on the general usefulness of sur- 
vey procedures and data. Other details of Zeigel’s study are re- 
ported in Chapter XV, on the training and supervision of 
research workers. 

The normative-survey method is not notably forward-look- 
ing, but it may be of service in this direction. It may reveal 
practices or conditions which are well above the average, repre- 
senting advanced thinking and administration; the worker 
should always be alert to discover such. The method is also 
helpful because it tends to focus attention on needs that might 
otherwise remain unobserved; many a city does not realize 


_ TW. H. Zeigel, Jr., Research in Secondary Education. U. S. Office of Education Bul- 
letin, 1932, No. 17; National Survey of Secondary Education Monograph, No. 15, 
Washington, 1933. Pp. 72. 
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the unsatisfactory conditions in some of its school buildings 
until a building survey reveals them. Again, the normative 
method may call attention to current trends® and permit 
people to evaluate and direct these new tendencies which are 
taking shape. The normative attack is not essentially forward- 
looking in itself, but it may well perform an important func- 
tion in giving pertinent data to persons who are forward- 
looking. 

The normative-survey method is not suited to the testing of 


various principles under laboratory conditions; it may, how- , 


ever, afford evidence of the results of certain tests under prac- 
tical conditions. It offers limited opportunity for the revelation 
of fundamental laws in anything like a pure form, but may 


provide an invaluable check upon laws that have been dis- 


covered in isolation from the normal complex of social forces. 
On certain laws of distribution, field study may yield direct 
evidence, which cannot be obtained from any other source. 
Social scientists are beginning to regard various areas of habi- 
tation as essentially natural laboratories for their purposes, 
and they are directing techniques to the analysis of these 
phenomena. The normative-survey attack does not, however, 
characteristically penetrate deeply into interrelationships. It 
stops with the disclosure of facts and a suggestion of relatively 
prominent connections between these and apparent causes. The 
full analysis of complex interrelationships in natural situations 
is discussed in Chapter X which deals with certain methods 
especially adapted to a study of causal relationships. 

The foregoing remarks on the value of survey studies will 
assure the research student that the type of information which 
the normative-survey method produces is in wide demand and 
is capable of rendering important service. They do not mean 
that the method is either a superior or an inferior type of re- 


8 For example, see Clara W, H. Gould, “Recent Biography:*An Analytical Study with 
a View to Determining the Essential Similarities and Differences in Subject Material and 
Treatment in Modern Biographical Writing for Adults, Young People in the Ninth 
Grade, and Children Below the Ninth Grade.” Master’s thesis. Cleveland: Western 
Reserve University, 1932. Pp. 86. 


294 EDUCATIONAL RESEARCH 


search. It is true that scientists do not generally regard it 
as a very high order of research, but that is of little conse- 
quence. Research is not necessarily carried on as an intellectual 
pastime of scientists; it frequently has important practical 
questions to answer, and it must be appraised in the light of 
its effectiveness in answering these questions. There is no 
“best” type of research in general; each kind may be “best” 
for a particular purpose, and the survey type has its own field 
of particular usefulness. In this field, as in any other, all 
degrees of quality are possible. On the one hand, survey studies 
may be so trivial, so poorly conceived or narrowly carried out, 
or so lacking in penetrating analysis, that they fail to rise 
above the level of clerical routine and can make no claim upon 
the name “research.” On the other hand, one may exercise 
great ingenuity, skill, and insight in planning and prosecuting 
his study so that it not only serves its immediate purpose but 
may also serve to set new standards for research in the area 
involved. 

The normative-survey attack fits into the total research 
scheme in several ways. It may represent the reconnaissance 
stage of research which is entering a new area, or it may repre- 
sent a specific interest in current conditions within an area that 
has long since been thoroughly explored and developed by 
research. If one watches the development of research in some 
new field through a series of stages, he will note that the 
early attacks represent a sort of general “getting acquainted” 
with the field—How far does it extend? What are the phe- 
nomena like? What are the classes or kinds into which the 
phenomena can be put? How many cases are there of each 
class? etc. Later on, as the worker gains familiarity with the 
characteristics of the field, he will attack the more detailed 
aspects of the area and make more refined studies, usually 
involving cross-classification, association, and cause and effect. 
The early investigations bear a genetic relation to these later 
studies just as the early efforts of the infant to talk bear to 
his later mastery of the intricate possibilities of purposive 
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speech. Although the exploratory studies in a new field may 
take the form of survey attacks and may be followed by re- 
search of a different sort that sheds additional light on the 
facts and relationships of the field, this does not indicate that 
such a réle represents the complete sphere of service of the 
survey techniques. On the contrary, the survey attack is always 
appropriate when information concerning current conditions is 
desired in any field, however well explored, in which there are 
changes of condition or changes of population frequency from 
time to time. j 

Kinds of normative-survey research, The normative-survey 
method of research finds expression through a variety of tech- 
niques. Just as there are different kinds of facts which one may 
wish to gather and different kinds of sources for these facts, so 
there are many procedures which have been developed to meet 
the heeds of these different purposes and conditions. Thus, when 
one desires facts from people about themselves, he may inter- 
view them, or he may ask them to fill out an information card, 
to supply answers to a questionnaire, to submit to a test or 
examination, or to produce something evidencing their skill or 
interest. For other purposes or other fields of work, appropriate 
methods and techniques are available. 

In all, six different types of normative-survey research are 
recognized in the present treatise, namely, (1) survey testing, 
(2) questionnaire inquiries, (3) documentary frequency 
studies, (4) interview studies, (5) observational studies, and 
(6) appraisal procedures. Because of the amount of material 
only the first three of these types are treated in the present 
chapter, the other three being discussed in the following chapter. 

In treating these types of normative-survey research, four 
general purposes will be maintained: (1) to show the different 
educational fields which may be studied by this method; (2) to 
illustrate data-gathering techniques which are applicable to 
these different fields; (3) to bring out the characteristics of 
the different types of normative-survey procedures, especially 
of the instruments which are used; and (4) to introduce typi- 
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cal studies which will acquaint the student with actual research 
in each field and provide suggestions for further work of his 
own. The discussions of the various types of survey research 
are followed by directions and suggestions for applying the 
procedures in one’s own study, and where special instruments 
or schedules are called for, the preparation of these is treated. 

Such purposes cannot be realized fully within the compass 
of a single volume, when entire books are devoted to several 
of the individual attacks. It will therefore be the purpose of 
this treatise to furnish the student with an overview of the 
field and to introduce in the footnotes enough references to 
the best literature on the subject so that he can become con- 
versant with each type of attack. This general purpose will 
be especially evident in the descriptions of preparing instru- 
ments, which are relatively brief. Thus, although the discus- 
sion should provide all students with a general orientation and 
background in normative-survey investigation, those students 
who expect to use any particular procedure must plan to make 
a more intensive study of that procedure by following up the 
wealth of literature given in the footnotes. 

The reader should remember that the types of data-gathering 
procedures which are characteristic of the normative-survey 
method are also employed in studies which depend chiefly on 
other research procedures. For example, experimental work, 
which is the third research method to be treated in this book, 
ordinarily calls for the measurement of groups of persons by 
tests, unless it belongs strictly to the laboratory type; ques- 
tionnaires, check-lists, direct observation, and other procedures 
may also be involved. Case work and genetic studies depend 
largely on measurements and interviews, and so on. The de- 
scription of these techniques of gathering data will, therefore, 
be of interest, nef only to those who are contemplating use of 
the normative-survey method, but to those who are expecting 
to engage in any kind of quantitative research. 

It may be appropriate to point out once more that there is 
a difference between research and the routine use of a tech- 
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nique. Research embodies considerable care in the formula- 
tion of the problem, and in the determination of the best pro- 
cedures for attacking the problem. It always involves a con- 
siderable element of novelty. It will be borne in mind that 
such a distinction does not imply that research is necessarily 
superior to any other level of fact finding. The complexity of 
one’s work should be dictated by the purpose for making the 
study and by the conditions of work. Immediate needs may 
often be met more expeditiously and sometimes more ac- 
curately by a routine study than by an elaborate special 
study. The distinction between research and less exacting 
levels of work is made in order that the meaning of the term 
research may be kept definite, and this is done without any in- 
tention of derogating from the value of less formal undertak- 
ings when they are appropriate to the needs and the conditions 
of work, In fact, some of the illustrations used in con- 
nection with the six types of normative-survey research which 
follow are not research in the full sense, but they serve never- 
theless to bring out particularly well the characteristics of the 
technique under discussion. When such studies find their place 
as a step in a large pattern of problem solving, they become 
a legitimate part of research. 


I, STUDIES EMPLOYING SURVEY-TESTING PROCEDURES 


Perhaps the commonest example of normative-survey re- 
search in education is survey testing. By survey testing is 
meant simply the testing of a group of children (or adults) 
to ascertain the prevailing condition with respect to the traits 
measured by the test. A teacher may give tests to the pupils 
in her class to discover the general level of the class per- 
formance. Similarly, tests may be given to the pupils in a 
school building, in a city, county, state, group of states, or 
larger area, and some testing surveys have been international.? 


9See for example, V. A. C. Henmon, Achievement Tests in the Modern Foreign 
Languages. Publications of the American and Canadian Committees on Modern Lan- 
guages, Vol. V. New York: The Macmillan f9., 1929. Pp. 363. 
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The primary purpose of such testing is not to reveal detailed 
facts about any particular person individually, as in the case 
study, but rather to indicate the prevailing conditions through- 
out the group.’ It is true that diagnostic and prognostic uses 
of tests (in connection with the causal-comparative, correla- 
tion, and case methods of research, described in Chapter X) 
emphasize study of the performances of individuals. To be 
considered research, survey testing, of course, must be con- 
ducted with reference to some clearly defined problem, such 
as the common errors of pupils in a particular school subject at 
a given grade level. 

In discussing survey testing, various uses will be described 
without thought of suggesting that these are normally un- 
dertaken as research projects. These uses are mentioned to 
bring out the breadth of the applications of survey testing, and 
to suggest the wide field in which research may be under- 
taken. The student who is contemplating research in this 
area may choose any suitable aspect of the field for investi- 
gation, including practical use as a data-collecting instrument 
in seeking an answer to some significant question, or actual 
study of the tests themselves (characteristics, construction, or 
methods of administering). In addition to these uses of survey 
testing, one may employ tests to furnish measurements for 
some other method of research, such as experimental research. 
There is no sharp dividing line between normative-survey 
research and historical or experimental or causal-comparative 
research, and the treatment of survey testing—and of the 
other techniques in this and the following chapter—is written 
with the thought that the technique described may be of serv- 
ice in other forms of research as well as in normative-survey 
research. The particular manner in which such a technique 
will fit into a research study depends upon the particular 
problem that is chosen and the way in which it is to be treated. 


10 See further notes on the purpose of survey testing, p. 460 of the Review of Edu- 
cational Research, V (December, 1935), No. 5. 
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Judgment must therefore be exercised in adapting the various 
techniques to the research problem that is undertaken. An 
examination of the studies cited should be helpful in suggest- 
ing ways of making this adaptation. 

Achievement testing. It is now somewhat common for 
achievement tests?! to be given to entire school systems by 
some central agency, sch as a bureau of research ™ in the 
superintendent’s office. Intelligence tests are frequently given 
by the same agency. A number of the larger school systems are 
constructing achievement tests of their own, based upon the 
courses of study which have been adopted for that particular 


11 Gertrude H. Hildreth, A Bibliography of Mental Tests and Rating Scales. New 
York: The Psychological Corporation, 1933. Pp. 242. This is the most comprehensive 
bibliography at the time of the present writing, for both scholastic achievement tests 
and intelligence tests. It should be supplemented for later tests by use of the follow- 
ing sources, (a) to (e). References (f) and (g) are included for those who desire 
earlier bibliographies. 

(a) Oscar K, Buros, Educational, Psychological, and Personality Tests. Published 
annually by the School of Education, Rutgers University, New Brunswick, New Jersey. 
This is a list of new tests that have appeared during the year, classified by subject, and 
indexed by title and by author. Information is given on grade range, number of forms, 
cost, time required, etc. References to articles on the tests are also given. This bibliog- 
raphy is designed as an annual supplement to Hildreth’s basic work, referred to above. 
The first issue of this publication appeared in 1935, covering tests for 1933 and 1934. 
Pp. 44. so¢. It. was Number Seven in the Studies in Education series of Rutgers Uni 
versity. 

(b) The Education Index. A current record of new tests of all kinds will, be found 
under the caption, “Tests and Scales,” in the Check-List; this Check-List is printed in 
the front of the monthly paper-bound issues, To find a particular test when the author, 
or title is known, look for that name in the main alphabet of the Education Index, in 
either the monthly or cumulative issues. To find tests in a certain subject, look under 
that subject for the subhead, “Tests and Scales.” For a suggestion of other subjects 
which may be related, consult the cross reference list at the end of the main topic, 
“Tests and Scales.” For literature discussing the characteristics and use of various 
tests, see the main topic, “Tests and Scales.” 

(c) Educational Research Bulletin, of the Bureau of Educational Research, Ohio State 
University, This periodical reviews new tests which appear, somewhat after the fashion 
‘of book reviews. 

(d) Watch the subhead, “Bibliography,” under the topic ‘Tests and Scales” in the 
Education Index for the appearance of any new bibliographies of tests. 

(e) For references to articles which evaluate and discuss the uses of various achieve- 
ment tests, see footnote 32. 

(f) Henry Lester Smith and Wendell William Wright, Second Revision of the Bibli- 
ography of Educational Measurements, Bulletin of the School of Education, Indiana 
University, IV (November, 1927), No. 2. Bloomington, Ind.: Bureau of Codperative 
Research, 1927. Pp. 251. 

(£) B. R. Buckingham, Margaret Doherty, and Josephine MacLatchy, Bibliography 
oj Educational and Psychological Tests and Measurements. U. S. Bureau of Education 
Bulletin, 1923, No. 5s. Washington, 1924. Pp. 233. 

12See for example H. B. Chapman, Organized Research in Education with Special 
Reference to the Bureau >f Educational Research. Bureau of Educational Research Mono- 
graph, No. 7. Columbus, Ohio: Ohio State University, 1927- Pp. 221, 
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system. The achievement-testing program ** of a central bu- 
reau may cover the same subjects and the same grades year 
after year, or variation from year to year may be provided in 
subjects, grades, or tests. 

The results of such programs enable the superintendent to 
know whether the pupil performance throughout the school sys- 
tem is, on the average, comparable with the average level of 
performance of other years, or of other school systems, with 
reference to the particular aspects of performance which are 
tested. If the results do not indicate a satisfactory level of 
work, the superintendent will make various other analyses de- 
signed to reveal the factors that are responsible for the un- 
satisfactory performance. The routine administration of tests 
is not to be regarded as research, but the employing of tests 
in the thorough analysis of a problem may constitute research 
of an important kind. Achievement testing is called for in 
many different kinds of research. 

Testing for appraisal. In the earlier testing literature, and 
even in that of more recent date, it has been stated that 
achievement tests could be used to appraise the work of teach- 
ers, of principals, and even of supervisors. Achievement test 
results are not to be ignored in judging the work of a teacher, 
but it must be realized that the teacher’s influence is only one 
factor among many conditions which determine the level of 
the child’s performance.*° For example, there are such addi- 
tional sources of influence as native brightness, physical vigor, 
home background, recreational habits and environment, and 
previous training and experience—all contributing factors of 
great potency in determining the present performance of pu- 

18 Clifford Woody and Paul V. Sangren, Administration of the Testing Program. 
Yonkers-on-Hudson, N, Y.: World Book Co., 1932. Pp. 397. “Designed to serve as a 
manual of instruction for those having the responsibility of directing a testing pro- 
PIa Cales Wi Knudsen, Evaluation and Improvement of Teaching, Chapter VII, 
“Evaluation of Instruction by Means of Standardized Tests,” pp. 285-320. Garden City, 
New York: Doubleday, Doran and Co., Inc., 1932. Includes a bibliography. 

15A. S. Barr, “Measurement in the Supervision and Improvement of Teaching,” 
Twenty-second Conference on Educational Measurement, pp. 25-31, Bulletin of the 


School of Education, Indiana University, XII (December, 1935), No. 1. Bloomington, 
Ind.: Bureau of Codperative Research. 
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pils.° Unless one secures reliable and comprehensive evidence 
on these other sources of influence, he has little right to at- 
tribute test results to the superiority or inferiority of a pat- 
ticular teacher or group of teachers. Test results from a class 
may be taken as suggestive evidence of the quality of teaching, 
to be confirmed or refuted by other lines of evidence, but never 
to be used as the sole or even the principal criterion. One 
should, therefore, be careful in choosing research problems in 
this area, with the intent of relying upon test results alone. 

Some may consider the possibility of using test results as a 
means of rating different educational institutions. For example, 
the administrative officers of a city school system might desire 
to know the relative standing of the various schools in the 
city on the basis of their educational effectiveness, so that their 
practices could be examined and the best ones established in 
other schools that were doing less effective work. Attention 
will be given to this problem of rating different educational 
institutions in connection with the topic of “Appraisal” in the 
following chapter. It may be pointed out in advance of that 
discussion that the abilities which we measure by tests con- 
stitute only one of many sets of desirable educational prod- 
ucts; furthermore, we cannot be certain how large a part 
the school has played in the production of even these outcomes. 
As in the case of judging the work of teachers, test results 
have a certain significance, but without a knowledge of the 
many selective factors which affect the pupil group in any 
school and without an appraisal of many dynamic influences 
in their daily lives, one will hesitate to say that test results 
are even presumptive evidence of the quality of work being 
done in any particular school. Research in this area must there- 
fore consider more than test results. 

Not infrequently the results of city-wide survey testing are 
made available to principals and teachers for an analysis of 
their schools and classes. One must be cautious not to put 


416 J. B. Maller, ‘Economic and Social Correlatives of School Progress in New York 
City,” Teachers College Record, XXXIV (May, 1933), 655-79. 
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survey tests to a more refined use than is appropriate. Although 
test makers have in general endeavored to construct tests that 
will yield reasonably accurate scores for an individual child, 
it must be understood that not every test—and particularly 
not every survey test—provides for a sufficient sampling of 
performance so that a single individual is reliably described. 
A survey test may afford an average for a large group, which 
is highly dependable,” and thus serve the superintendent’s 
and principal's interests without giving the teacher anything 
that is more than suggestive for her class or for an individual 
pupil. Further, it must be recognized that the purposes and 
interests of a teacher and an administrator frequently differ, 
and that the satisfaction of the teacher's interests may require 
a very different kind of test. Survey testing may easily be 
abused through putting interpretations on the results that the 
tests do not warrant. Here, as in all research, one must be 
careful to distinguish between what his results actually show, 
and what they appear to indicate at first glance. Careful inter- 
pretation of “facts” is as important as’careful work in gather- 
ing them. 

State and regional achievement testing. There are a num- 
ber of regular achievement testing projects that cover large 
areas, such as States, regions, or the country as a whole.® Al- 
though these undertakings normally contemplate considerable 
use of the results for individual pupils, the projects, neverthe- 
less, have obvious survey characteristics, and they are referred 
to here for that reason. In certain of the states achievement 
tests are given more or less regularly by the state department 
of education to schools throughout the state. Such practices in 
six of the states have been described by the Research Division 
of the National Education Association.2® 

17 In any case care must be exercised in the interpretation of test scores. See the 
later discussion on this point. ry 

18 Paul V. Sangren, ‘State and National Testing Programs,” in ‘Educational Tests 
and Their Uses,” Review of Educational Research, V (December, 1935), No. 5, 456-8. 

19 “State-Wide Achievement Testing Programs,” in “Estimating State School Eff- 
ciency,” Research Bulletin of the National Education Association, X (May, 1932), 


No. 3, 97-103. Includes a bibliography of forty references on state-wide achievement 
testing. : 
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State testing programs especially for high schools are de- 
scribed by Segel.?° He lists eighteen states; South Carolina has 
recently joined the list, giving its first state-wide tests in May, 
1936.2 Some of these programs extend also to high-school 
pupils of other states. A number of states have testing pro- 
grams for graduation from the eighth grade. These various 
state programs are usually conducted by bureaus of research 
or measurement in the state department of education, in the 
department of education of the state universities, or in some 
teacher-training institution. In some cases the programs are 
carried on by a voluntary organization. 

Several agencies conduct achievement testing on a national 
scale. The Bureau of Educational Measurements of the Kansas 
State Teachers College at Emporia conducts a “Nation-Wide 
Every Pupil Scholarship Test” in January and April of each 
year, which may extend over some forty-two states and in- 
volve around a million tests. This enterprise has been going 
on since 1925. The stated purpose of the project is “the pro- 
motion of scholarship.” The Bureau makes its own tests for 
these occasions and supplies them to elementary and high 
schools that desire them for a and 2¢ apiece, respectively. 
Each school gives and scores its own tests and sends the re- 
sults to the Bureau, which then calculates norms for the na- 
tion, and for many of the individual states, and returns these 
to the schools which participated so that they may compare 
their averages with the norms. 

The Public School Publishing Co., of Bloomington, Ill., has 
also engaged in a nation-wide testing program annually since 
1924, with possible follow-up tests later in the year; these 
programs embrace both elementary and high schools and are 
perhaps even more extensive than the Kansas project. In addi- 
tion to these are three agencies which confine their attention 
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to high-school pupils. College-entrance examinations have long 
been given on a national basis by the College Entrance Ex- 
amination Board, but there are now other agencies, notably 
the Educational Records Bureau of New York City and the 
Coéperative Test Service, which are interested in testing 
pupils in any grade of the high school.*° 

It should be said that routine testing is not in itself re- 
search, whether it is on a national basis or in a single class- 
roum. Reference has been made to these large testing pro- 
grams because they represent procedures for gathering data 
which the normative-survey method employs; they may serve 
as a source of data for some one who desires to make use of 
their results for more analytical research purposes; and, 
finally, survey testing over considerable areas has become an 
important part of large-scale research undertakings which may 
include other kinds of investigation. Such surveys are often 
the inspiration for research studies. For example, H. A. Toops 
has made many researches from data gathered through the 
tests of the Ohio College Association. Numerous other similar 
examples can be found. 

Achievement testing in school surveys. Survey testing of 
educational attainment has become a large and well-estab- 
lished part of school surveys. Caswell °? traces briefly the his- 
tory of the use of tests in surveys. Whereas in 1911 the 
appraisal of instruction was based on an inspection of the teach- 
ing by a surveyor, achievement tests were given some recogni- 
tion in a 1913 survey, and considerably more use was made 
of them in r914 in the Butte, Montana, survey. In the Cleve- 
land, Grand Rapids, and the St. Louis surveys of 1916 and 1917, 
under the influence of Charles H. Judd, achievement testing 
was raised to a height that has set a standard for all subsequent 
surveys. 

Achievement testing is now a regular part of school surveys.** 

22 Hollis Leland Caswell, City School Surveys. Contributions to Education, No. 358. 
New York: Teachers College, Columbia University, 1929. Pp. 130. 


23 For surveys which have employed standard achievement tests in various school 
subjects see the topic, “The Measurement of Instruction,” Topical Analysis of 234 
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The testing in these cases is of the survey type,* in which 
emphasis is placed upon group results, and the value of the 
instruction is appraised accordingly. In some surveys an at- 
tempt is made to analyze the results in detail so that construc- 
tive suggestions for the improvement of the practices can be 
offered.?* 

Surveys of school systems have extended beyond the limits 
of cities, to include counties, states, and the entire country. 
Regardless of the territory included, testing plays an important 
part.2* More will not be said about surveys at this point, as 
they are discussed in the next chapter under the topic, 
“Appraisal procedures.” 

It will be clear from what has already been stated that there 
is some question concerning the satisfactory use of the usual 
achievement tests as the sole basis for drawing conclusions 
about instruction.2* The customary tests do not measure many 
of the objectives that the better teachers are striving to de- 
velop,* and, conversely, good results of the formal kind can be 
achieved by the use of teaching procedures which in many 
cases do much harm, such as an overemphasis upon drill, unen- 
lightened rote memory, continual competition, etc., not to 
mention a corresponding lack of emphasis upon broader aspects 
of growth. The older idea, that the teacher who secures good 


School Surveys, by Henry L. Smith, pp. 75-82. Bulletin of the School of Education, 
Indiana University, III (March, 1927), No. 4. Bloomington, Ind.: Bureau of Co- 
operative Research, Pp. 111. 

24 Walter Crosby Eells, “Use of Standard Tests in 72 Published School Surveys,” 
School Life, XIV (May, 1929), 168-9. 

25 See for example the Report of the Survey of the Schools of Chicago, Illinois, 
especially Vol. II, “Fitting the School to the Pupil,” pp. 3-141, by Paul R. Mort, 
Wendell W. Wright, and W. B. Featherstone. Or, the same topic, and other sections, 
in Vol. V, “Summary of Findings and Recommendations,” pp. 72-86, by George D. 
Strayer, New York: Teachers College, Columbia University, 1932. 5 vols. 

26 See the reference in footnote 19. " 

27 Conclusions and Recommendations of the Commission. Report of the Commission on 
the Social Studies of The American Historical Association, Investigation of the Social 
Studies in the Schools, pp. 96-ror. New York: Charles Scribner’s Sons, 1934. Pp. 168. 

A. S. Barr, An Introduction to the Scientific Study of Classroom Supervision, Chapter 
III, “The Survey of the Products of Learning,” pp. 62-115. New York: D. Appleton- 
Century Co., 1931. Pp. 399. y P 

28 Ralph W. Tyler, “Evaluation: A Challenge and an Opportunity to Progressive 
Education,” Educational Record, XVI (January, 1935), 122-31. The appraisal of edu- 
cational outcomes other than the formal ones is treated in the next chapter of this book 
under the topic, “Observation of Behavior’ and ‘Appraisal Procedures,” as well as in 
the section on “Questionnaire Inquiries” which follows. 
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performance by her pupils on formal tests is likely to be a 
superior teacher in all respects, is giving way to a more dis- 
criminating insight which recognizes that good test results 
are sometimes secured at too high a price. 

Survey testing in complex studies. Survey testing of 
achievement not infrequently forms a part of large studies 
which include many people and deal with a number of aspects 
of a complex problem. In such cases tests may be constructed 
especially for the occasion, or existing tests may be used with 
or without modification. The Modern Foreign Language 
Study *° furnishes one example of such use. The Nation-Wide 
Study of English Usage *° was based almost entirely on survey 
testing. The Payne Fund Studies of motion pictures used tests 
for important phases of the inquiry.*t Such studies as these 
furnish additional examples of the way in which survey testing 
may be of importance. 

Uses of survey tests in connection with experiments will be 
apparent in the treatment given in Chapter IX on the experi- 
mental method. Tests also form the basis of much individual 
and statistical work described in Chapter X.** 


29 This study is published in seventeen volumes; see for example: Ben D. Wood, 
New York Experiments with the New-Type Modern Language Tests. Publications of the 
American and Canadian Committees on Modern Languages, Vol. I. New York: The 
Macmillan Co., 1927. Pp. 339. See also the summary volume by Robert H. Fife. 

80L. J.-O’Rourke, Rebuilding the English-Usage Curriculum to Insure Greater Mas- 
tery of Essentials, Washington: The Psychological Institute, 1934. Pp. 98. See also 
Paul S. Achilles, ‘“Nation-Wide Surveys in Reading, English Usage, and Vocabulary 
Launched by the Psychological Corporation,” School and Society, XXXIV (August 15, 
1931), 233-5. 

81 P. W. Holaday and George D. Stoddard, Getting Ideas from the Movies, New 
York: The Macmillan Co., r933. Pp. «02. Briefly described in W. W. Charters, Motion 
Pictures and Youth, pp. 7-11, 1933. (Note: The volume by Charters is published both 
separately and under the same cover as the volume by Holaday and Stoddard.) 

832 For further uses and studies of achievement tests, see “Educational Tests and 
Their Uses,” Review of Educational Research, V (December, 1935), No. 5; also the 
earlier number, III (February, 1933), No. 1, and later numbers as they appear in the 
three-year cycle, 

A series of annual reviews of studies ^f educational tests has appeared in the 
Psychological Bulletin, beginning with the July, 1926, number (Vol. XXIII). These 
appeared in the July number of each succeeding year through 1929, then in Jnne, 1930, 
(none in 1931), in February, 1932, and in July, 1933, 1934, and 1935. Presumably the 
policy will be continued. 

For a bibliography of both tests and studies of tests, covering the earlier period of 
aiterature in this field, see: B. R. Buckingham, Margaret Doherty, and Josephine Mac- 
Latchy, Bibliography of Educational and Psychological Tests and Measurements. U. S. 
Bureau of Education Bulletin, 1923, No. 55. Washington: 1924. Pp. 233. 

See also the topics, “Achievement Tests,” “Educational Measurements,” “Objective Ex- 
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Testing in foreign countries. European countries, especially 
those on the continent, do not share the American attitude 
toward testing to any large degree. In general they are actu- 
ated by a different basic philosophy, which expresses itself in 
other interests. Intelligence tests are likely to be used where 
we would use achievement tests. There has, however, been a 
limited amount of investigation using achievement tests, and 
there has been some survey testing. Educational testing in 
England, France, Germany, and China has been summarized 
in the Review of Educational Research.* The student who 
desires to get in touch with expressions of the European atti- 
tude at first hand will be interested in reading reports of 
international conferences on measurement.** These diverse at- 
titudes towards measurement should suggest a number of prob- 
lems for investigation. 

References to the use of intelligence tests in foreign countries 
will be found in footnotes 33 and 43 of this chapter, and refer- 
ences to the use of personality tests in foreign countries are 
listed by Maller (footnote 45). 

Intelligence testing. Intelligence tests purport to measure 
the factor of native brightness, or capacity for intellectual 
adaptation, as it varies from individual to individual. It is gen- 
erally recognized that these tests are not free from the influence 
of formal training and general experience, so that scores repre- 
sent a composite of native and experiential factors.’ To-day 


aminations,” and “Tests and Scales,” in the Education Index, and the topic, “Educational 
Tests,” in the Bibliography of Research Studies in Education, issued annually by the 
United States Office of Education. 

88 Charles C. Weidemann, W. J. Osburn, and Louise Mahone, ‘Educational Tests 
and Measurements in China, England, France, and Germany,” pp. 443-54, in “Educational 
Tests and Their Uses,” Review of Educational Research, V (December, 1935), No, 5. 
Bibliography of 204 references. 

34 Paul Monroe, editor, Conference on Examinations, under the auspices of the 
Carnegie Corporation, the Carnegie Foundation, and the International Institute of 
Teachers College, Columbia University, at Eastbourne, England, May, 1931. New York: 
Teachers College, Columbia University, 1931. Pp. 316. 

A second conference (London, July, 1933), and a third one (Folkestone, England, 
June, 1935), have been held. These conferences and their reports are described by I. L. 
Kandel, in “The International Examinations Inquiry,” Educational Record, XVII (Janu- 
ary, 1936, Supplement No. 9), 50-9. Also, more briefly, in the Journal of Educational 
Research, XXIX (December, 1935), 314-5. 

35 For a pointed but comprehensive statement of various influences, see Frank N 
Freeman, Mental Tests, Chapter XVII, “Interpretation of Intelligence Tests,” pp- 
444-75. Boston: Houghton Mifflin Co., 1926. Pp. 503. 
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good intelligence tests are, however, the best means available 
for reflecting, to as large a degree as possible, the factors of in- 
herited capacity. In that rôle they are highly useful in many 
ways.*° 

In individual school systems, intelligence tests are largely 
employed as a basis for dividing large classes into relatively 
homogeneous sections," as an aid in the diagnosis and adjust- 
ment of individual children, in educational and vocational 


For more recent, technical discussions, see Harry J. Baker, “Intelligence and Its 
Measurement,’ Chapter I, pp. 187-98, in “Psychological Tests,” Review of Educa- 
tional Research, V (June, 1935), No. 3. Also, Katharine B. Greene, “General Intelli- 
gence and Its Measurement,” Chapter I, pp. 274-83, in “Tests of Intelligence and Apti- 
tude,” Review of Educational Research, I (October, 1932), No. 4. 

86 For illustrative treatments of various uses and studies of intelligence tests, see 
the following: 

Gertrude Hildreth, “Applications of Intelligence Testing,” Chapter II, PB. 
in “Psychological Tests,” Review of Educational Research, V (June, 1935), 

Paul R. Mort and W. B. Featherstone, “The General Uses of ERR Tests,” 
and “The Uses of Psychological Tests for Atypical Groups,” Chapters V and VI, pp. 
300-20, in “Tests of Intelligence and Aptitude,” Review of Educational Research, IL 
(October, 1932), No. 4. 

Rudolph Pintner, “Intelligence Tests.” A series of annual reviews of studies of intelli- 
gence tests and their uses, in the Psychological Bulletin. Parallels the series on edu- 
cational tests; see footnote 32. 

, Intelligence Testing, Part III, pp. 225-541. New York: Henry Holt and Co., 
ad ed., 1931. Pp. 555. 

Paul L. Boynton, Intelligence: Its Manijestations and Measurement, Chapters IX and 
X, pp. 337-451. New York: D. Appleton-Century Co., 1933. Pp. 466. 

Henry E. Garrett and Matthew R. Schneck, Psychological Tests, Methods, and Re- 
sults, Part II, Chapter V, “Some Applications of Psychological Tests,” pp, 185-224. 
New York: Harper and Bros., 1933. Parts I and II. 

Frank N. Freeman, Mental Tests, Chapter XIV, “The Educational Uses of Tests,” pp- 
365-98. Boston: Houghton Mifflin Co., 1926. Pp. 503. 

See the topics, “Intelligence Testing,” and “Intelligence Tests,” pp. 36-7 in the 
Subject Index of The Personnel Bibliographical Index, by W. H. Cowley. Columbus: 
Bureau of Educational Research, Ohio State University, 1932. Pp. 433. 

In the Education Index, see the topics: “Intelligence,” “Intelligence Quotient,” ‘“In- 
(elligence Tests,” ‘Psychological Tests,” and “Achievement Quotients.” In the Bibliog- 
raphy of Research Studies in Education, issued annually by the United States Office of 
Education, see the topic (in the index), “Psychological Tests.” 

In the annual Psychological Index, see section XI, No, 2, ‘Mental Tests,” which in- 
cludes character and personality tests also. 

87'Paul T. Rankin, “Pupil Classification and Grouping,” Chapter III, pp. 200-20, in 
“School Organization,” Review of Educational Research, 1 (June, 1931), No. 3- 

“Bases of Grouping,” pp. 384-5, in “School Organization,” Review of Educational 
Research, IV (October, 1934), No. 4. 

Robert T. Rock, Jr., A Critical Study of Current Practices in Ability Grouping, Chap- 
ter III, “Intelligence as a Basis of Classification,” pp. 30-62. The Catholic University 
of America, Educational Research Bulletins, IV (May-June, 1929), Nos. s and 6. 
Washington: The Catholic Education Press, 1929. Pp. 132. Bibliography of 110 refer- 
ences. 

Harl R. Douglass, “Certain Aspects of the Problem of Where We Stand with Ref- 
erence to the Practicability of Grouping,” Journal of Educational Research, XXVI 
(January, 1933), 344-53. 

See also the references in the preceding footnote 


par 214, 
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guidance," and in other ways. Many of the larger cities have 
psychological laboratories or clinics for the study of children 
who are educationally or socially maladjusted.° The oppor- 
tunity for research in these areas is almost unlimited, though 
one will recognize that intelligence tests furnish only one line 
of evidence. 

Intelligence tests have received a great deal of use in the 
estimation of aptitude. Such prognosis is of course essential 
for guidance, both educational and vocational. Applications 
have been made to the estimation of general scholastic success, 
the probability of success in college, aptitude for particular 
high-school or college subjects, and future ability in various 
vocational fields.*® In this connection it will be borne in mind 
that educational achievement tests are also of value for pre- 
dictive purposes and are often used in this way. (Prediction, 
being based upon correlation, lies somewhat beyond the limits 
of normative-survey research. It is referred to in Chapter X.) 

Perhaps the largest use of intelligence tests has taken place 
in the field of scientific experimentation and study. A review 
of the references cited will show that most of them represent 


38 Edward L. Thorndike and Others, Prediction of Vocational Success. New York: 
Commonwealth Fund, 1934. Pp. 284. 

Harry J. Baker, ‘Vocational Aptitudes Tests and Their Application,” Chapter VII, 
pp. 321-4, in “Tests of Intelligence and Aptitude,” Review of Educational Research, II 
(October, 1932), No. 4. See also “Psychological Tests as a Predictive Device,” ‘‘Predic- 
tion of Academic Success in College,” and “Prediction of Teaching Success,” pp. 301-6, 
in this same isue. 

Ruth Strang, Personal Development and Guidance in College and Secondary School. 
New York: Harper and Bros., 1934. Pp. 341. (Contains a comprehensive bibliography 
of tests for educational guidance.) 

Herbert A. Toops, ‘Measures of Aptitude,” Chapter III, pp. 215-28, in ‘“Psycho- 
logical Tests,” Review of Educational Research, V (June, 1935), No, 3. 

See the topics, “Aptitude Tests,” “Vocational Guidance,” and “Vocational Tests,” in 
the Education Index, and the topics, “Tests and Scales: Aptitude,” and “Educational 
and Vocutional Guidance,” in the Bibliography of Research Studies in Education, issued 
annually by the U. S. Office of Education, 

The College Entrance Examination Board publishes annually a Report of the Commis- 
sion om Scholastic Aptitude Tests, which reviews the performance of candidates on its 
aptitude tests, and reports an analytical statistical study of the tests. Published by the 
Board, 431 W. 117th St., New York, N. y. 

See the six topics beginning with “Aptitude,” p. 16; “Techniques of Measuring 
Vocational Interests, and Vocational Success,” p. 63; and “Techniques of Predicting 
Vocational Aptitudes and Vocational Success,” p. 63; in the Subject Index of The 
Personnel Bibliographical Index, by W. H. Cowley. Columbus: Bureau of Educational 
Research, Ohio State University, 1932- Pp. 433- 


39 Gertrude Hildreth, Psychological Service for School Problems. Yonkers-on-Hudson, 
N. Y.: World Book Company, 1930. Pp. 317. 
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studies which have been made for research purposes. This is 
probably because intelligence testing lies primarily in the field 
of psychology, and psychologists are concerned chiefly with the 
development of their science rather than with immediately 
practical applications. This emphasis is somewhat in contrast 
to the stress in the field of educational or achievement testing, 
which has been almost entirely in the hands of educators, al- 
though probably in neither case can the emphasis be said to be 
disproportionate. 

A number of large-scale studies utilizing intelligence tests 
have been made. For example, surveys of intelligence have been 
conducted to ascertain the average intellectual level of pupils 
who enter college, of pupils who graduate from high school but 
do not enter college, of pupils who succeed in school and of 
those who do not, etc.*° Intelligence tests are among those given 
in state-wide testing programs,‘ particularly for the purpose of 
college entrance. Intelligence tests have also been used in school 
surveys,“ although to a much less extent than have achieve- 
ment tests. In foreign countries, intelligence tests have re- 
ceived a proportionally larger use than have achievement tests, 
being used often under circumstances for which achievement 
tests would be employed in America. Needless to say, 
the tests used in large-group studies are almost invariably 
group intelligence tests. 


40 William F. Book, The Intelligence of High School Seniors as Revealed by a State- 
Wide Mental Survey of Indiana High Schools. New York: The Macmillan Co., 1922. 
Pp. 371. 

Joseph R. Gerberich, “Gifted Pupils of the Towa High School Survey,” Journal of 
Applied Psychology, XIV (December, 1930), 566-76. 

Charles W. Odell, Are College Students a Select Group? Bureau of Educational Re- 
search Bulletin, No, 34. Urbana, Ill.: University of Illinois, 1927, Pp. 45. 

See the topic, “Intelligence of,” on PD. 36 of the Subject Index of Cowley’s Personnel 
Bibliographical Index (cited in footnote 39). 

41 “State-wide Coiperative Testing Programs,” pp. 219-21 in “Psychological Tests,” 
Review of Educational Research, V (June, 1935), No. 3. 

“The American Council on Education sponsors the construction of an annual edition 
of a psychological examination for use by colleges and universities in testing college 
entrants.” (Segel, p. 8, cited in footnote 20.) 

42 See the references listed on p. 79 of Topical Analysis of 234 School Surveys, cited 
in footnote 23. 

43 A discussion of intelligence testing in foreign countries will be found in the refer- 
ence given in footnote 33. Other references will be found scattered through ‘Psychologi- 
«al Tests,” Review of Educational Research, V (June, 1935), No. 3. 
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For a bibliography of intelligence tests which are available, 
see Hildreth and Buros and the other references given in 
footnote rz of this chapter. 

Personality testing. Personality constitutes the third large 
field, as the psychological aspects of human beings are 
commonly divided. It is somewhat commonly spoken of as 
“character and personality,” though in a strictly psychological 
sense the term personality may cover all of the response 
tendencies of the individual that are not already included 
under achievement and intelligence—and it is clear that it 
must also embrace some or even all of the characteristics 
in those fields. The line of demarcation is simply one of con- 
venience; it is impossible to draw a sharp dividing line between 
the different concepts. 

In the field of practical education it has long been recognized 
that personality traits are among the most important outcomes 
to be hoped for. They are, however, difficult of definition, 
analysis, and quantitative evaluation. The modern measure- 
ment movement has naturally given its first efforts to those 
things which are relatively easy to deal with, and in the volume 
of tests which have been produced and commercially distrib- 
uted in recent years there has been an emphasis that has 
tended to push the personality field into the background. The 
practical teacher and administrator have often taken a position 
in active opposition to the new testing movement for this 
reason. 

Up to the present time, instruments for survey testing in the 
field of personality, character, and adjustment have been in 
the formative stage and have not enjoyed widespread use. Most 
of the work has been a problem of developmental and promo- 
tional tryout. In this connection, mention should be made of 
the monumental pioneering work of Hartshorne, May, Shuttle- 
worth, and Maller, in the Character Education Inquiry.** Al- 

44 Hugh Hartshorne, Mark A. May, Frank K. Shuttleworth, and Julius B. Maller, 
Studies in the Nature of Character. Three volumes: Studies in Deceit, Studies in 


Service and Self-Control, and Studies in the Organization of Character. New York: 
The Macmillan Co., 1928-1930- 
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though much of this study was made with an emphasis on 
individuals and on correlation and employed other measures 
than tests, certain sections represent survey testing on a fairly 
large scale. 

It appears, however, that at the present time enough success- 
ful experimentation has been done in this field so that tests or 
other appropriate devices for evaluation are now ready for 
extended use." This does not mean that these procedures are 
perfect or “fool proof”; but neither are achievement and intel- 
ligence tests. The procedures which are appropriate are not 
confined to paper-and-pencil tests, but embrace a variety of 
techniques. For example, questionnaires, interviews, observa- 
tion, check-lists, and rating scales, as well as carefully pre- 
arranged social situations, are all employed in addition to 
paper tests. Some of these procedures depend to a large extent 
upon the skill and judgment of the individual doing the ap- 
praising; this means that they are not so satisfactorily done by 
the average person. On the other hand, it must not be over- 
looked that the best tests of intelligence are the individual 
ones, which require the skill and judgment of an expert. Cer- 
tain of the personality tests are as easily given as ordinary 
achievement tests, and are available for use in research studies 
as well as for regular purposes. 

Symonds has prepared a very readable, comprehensive 
treatise on the procedures in this field.** After an introduction, 
he treats of observation, rating methods, various types of ques- 


45 The bibliographies by Hildreth and Buros, listed in footnote 11, include tests of 
personality. The two books by Symonds, cited in footnotes 46 and 47, are also excellent 
sources, The references in footnote 48, dealing with studies of these tests, also contain 
lists, particularly the Review of Educational Research, IL (June, 1932), No. 3, by 
Goodwin Watson. See also footnotes 156-160 in Chapter VIII. 

J. B. Maller, “Bibliography of Character and Personality Tests.” New York; Insti- 
tute of School Experimentation, Teachers College, Columbia University, 1932. Mimeo- 
graphed. Pp. 53. A descriptive bibliography of some 300 tests, with notes on tests 
developed in England, Germany, France, and Russia. 

Henry E. Garrett and Matthew R. Schneck describe a number of representative per- 
sonality tests, and discuss their characteristics. See Psychological Tests, Methods, and 
Results, Part II, pp. 151-62. New York: Harper and Bros., 1933. 

For currently new tests, consult the annual bibliography by Buros cited in foot- 
note rr, and see the heading, ‘“Check-List: Tests and Scales,” in the front of paper- 
bound monthly issues of the Education Index. 

46 Percival M. Symonds, Diagnosing Personality and Conduct. New York: The Cen- 


tury Co., 1931. Pp. 602. 
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tionnaires for measuring adjustment, attitudes, and interests, 
tests of knowledge and judgment in social relationships, per- 
formance tests, physiological measures, interviewing, and meas- 
ures of the environment. These discussions deal with the meth- 
ods of carrying on the appraisal and report the results of various 
critical studies that have been made of the methods. In another 
book 47 Symonds lists various tests and other instruments that 
are available. He groups these under fifteen heads and gives 
information concerning the purpose, nature, reliability, and 
norms of each one, as well as references that deal with the use 
of the instruments. These two books are recommended as a 
readable, but thoroughly professional approach to this inter- 
esting field. 

There is already an extensive literature in this area, and the 
reader is referred to it ** for acquaintance with the develop- 


47 Percival M. Symonds, Psychological Diagnosis in Social Adjustment, New York: 
American Book Co., 1934. Pp. 362. See the appendix, pp. 171-346. 

48 Willard C. Olson, “General Survey of the Field of Character and Personality 
Measurement,’ Chap. V, pp. 242-44; ‘Measures of Character and Personality Through 
Conduct and Performance,” Chapter VIII, pp. 273-90; and Goodwin Watson, “Mental 
Hygiene and Emotional Adjustment,” Chapter VI, pp. 245-58; in “Psychological Tests,” 
Review of Educational Research, V (June, 1935), No. 3. 

Goodwin B. Watson, “Tests of Personality and Character,” Review of Educational 
Research, II (June, 1932), No. 3. 

Vernon Jones, “Character Education,” Chapter III, pp. 31-6, in “Special Methods 
and Psychology of the Elementary School Subjects,” Review of Educational Research, 
V (February, 1935), No. 1. 
and Lester A. Kirkendall, “Moral and Character Education,” Chapter XI, 
Dp. 507-9, in ‘Psychology and Methods in the High School and College,” Review of 
Educational Research, IV (December, 1934), No. 5. x 

H. J. Baker and Virginia Traphagen, The Diagnosis and Treatment of Behavior- 
Problem Children. New York: The Macmillan Co., 1935- Pp. 393. 

Percival M. Symonds and C. E. Jackson, Measurement of the Personality Adjustments 
of High School Pupils. New York: Teachers College, Columbia University, 1935. Pp. 121. 

See the studies in footnote 44, and those referred to by the books cited in footnotes 
46 and 47. 

A series of annual reviews of studies of character and personality measures was 
begun in the July, 1926, Psychological Bulletin. Reviews appeared in the July number 
each year through 1929, in June, 1930, (none in 1931), in February, 1932, and in the 
July numbers from 1933 on. These are comprehensive and valuable reviews. 

The Classroom Teacher and Character Education, Chapter VI, “The Teacher and In- 
dividual Pupil Guidance,” pp. 135-68. Includes a bibliography of problems. Seventh 
Yearbook, the Department of Classroom Teachers. Washington: National Education 
Assuciation, June, 1932. 

See the annual bulletins of the United States Office of Education entitled, Bibliography 
of Research Studies in Education, the section headed “Tests and Testing: Social Intelli- 
Bence—Tests and Scales,” or, “Tests of Social Adaptation.” 

Refer to the Education Index, for articles under “Character Analysis,” “Character 
Tests,” “Personality Tests,” “Interest—Tests and Scales,” “School Children—Adjust- 
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ments. Many studies have been made exploring the realm of 
personality and relating various degrees of adjustment or inter- 
est or other forms of adaptation to different factors in the 
environment or in the experience or nature of the individual. 
The relative newness of the field suggests that there are many 
opportunities for those who undertake research within this 
area. 

It is common psychological practice to include attitudes in 
the field of personality. Attitudes represent one of the more 
intellectual aspects of personality, and attempts to measure 
them have therefore been of the paper-and-pencil variety.“ 
Notable in this field has been the work of Thurstone and his 
students, who have prepared attitude scales on a number of 
social questions. These scales consist of a number of state- 
ments which have been assigned different scale values by a 
group of judges, and a person checks those statements with 
which he agrees. The basic theory of the method and the con- 
struction of the scales have been presented in a pamphlet °° 
as well as in numerous articles. 

Other instruments for indexing attitudes have been prepared. 
These are well described in the literature." Scales and their 


ment,” “Orientation Tests,” “Social Adjustment and Development,” “Social Intelli- 
gence—Tests and Scales,” and ‘“Behavior—Tests and Scales.” 

See the topic, ‘Techniques of Measuring,” p. 62, in the Subject Index of The Per- 
sonnel Bibliographical Index, by W. H. Cowley. Columbus: Bureau of Educational 
Research, Ohio State University, 1932. Pp. 433. 

49 Tests and scales for the measurement of attitudes are listed on pp. 165-8 of Gertrude 
H. Hildreth, A Bibliography of Mental Tests and Rating Scales. New York: The Psy- 
chological Corporation, 1933. Pp. 242. 

See also footnotes 51-54 for other sources of tests and scales, 

For currently new tests, see the annual bibliography by Buros cited in footnote r1, and 
see the heading, “Check-List: Tests and Scales,” in the front of paper-bound monthly 
issues of the Education Index. 

50L. L. Thurstone and E. J. Chave, The Measurement of Attitude. Chicago: Uni» 
versity of Chicago Press, 1929. Pp. 97. 

Henry E. Garrett and Matthew R. Schneck, Psychological Tests, Methods, and Results, 
Part II, pp. 139-41, and 148-9. New York: Harper and Bros., 1933. Parts I and II. 

Ralph H. Ojemann and Others, Researches in Parent Education, IIT. Studies in Child 
Welfare, X (October 15, 1934). New Series, No. 285, Iowa City: University of Iowa, 
1935. Pp. 391. 

George A. Lundberg, Social Research, Chapter IX, “The Measurement of Attitudes,” 
pp. 197-242. (References are given in Appendix A.) New York: Longmans, Green and Co., 
1929. Pp. 380. 

51 Percival M. Symonds, Diagnosing Personality and Conduct, Chapter VI, “Attitude 
Questionnaires,” pp. 215-38. Bibliography of thirty-one references. See also the topic, 
“Attitudes,” in the Index, for scattered references. New York: The Century Co., 1931. 
Pp. 602. 
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various uses are described in the Review of Educational Re- 
search," and they are included in the annual reviews of char- 
acter and personality tests which appear in the Psychological 
Bulletin.* Droba gives a review of measures of social attitude, 
including a bibliography of 104 titles.** Another source of ref- 
erences is the Education Index under the topic of “Attitudes.” 
“Thurstone Attitude Scales” may also be consulted in the 
Education Index for references on his scales. The study of 
any one or more of these sources will quickly give the student 
an acquaintance with what has been done in this field and with 
the methods of work. Other studies will be found in the annual 
United States Office of Education Bibliography of Research 
Studies in Education, by looking for “Attitudes” and “Tests— 
Attitudes” in the index. Measurement of attitudes is a rela- 
tively new field, but the literature has grown rapidly. 

The Payne Fund Studies of Motion Pictures and Youth 
afford several excellent illustrations of uses which can be made 
of attitude measures in scientific work. These uses are briefly 
reviewed in the summary volume by Charters," and are treated 
in more detail in the volumes reporting the several studies 5e 
which were concerned with attitudes. Uhrbrock reports an in- 
teresting investigation of attitude of employees,” which may 


, Psychological Diagnosis in Social Adjustment, pp. 226-7, 229-30, 247-62, 
305-6. New York: American Book Co., 1934. Pp. 362. 

See the topics, “Attitudes, Techniques of Measuring,” p. 18, and “Interests, Tech- 
niques of Measuring,” p. 37, and related topics, in the Subject Index of The Personnel 
Bibliographical Index, by W. H. Cowley. Columbus: Bureau of Educational Research, 
Ohio State University, 1932. Pp. 433. 

52 Goodwin B. Watson, “Social Attitudes,” Chapter VII, pp. 259-72, in “Psychological 
Tests,” Review of Educational Research, V (June, 1935), No. 3; and “Opinions, Atti- 
tudes, and Prejudices,” pp. 227-33, in “Tests of Personality and Character,” Review of 
Educational Research, II (June, 1932), No. 3- 

53 See footnote 48. 

54D. D. Droba, “Topical Summaries of Current Literature: Social Attitudes,” Ameri- 
can Journal of Sociology, XXXIX (January, 1934), 513-24- 

55 W, W. Charters, Motion Pictures and Youth, pp. 11-25. New York: The Macmillan 
Co., 1935. Pp. 66. 

56 Paul G. Cressey and Frederick M. Thrasher, Boys, Movies, and City Streets. New 
York: The Macmillan Co. In press. 

Frank K. Shuttleworth and Mark A. May, The Social Conduct and Attitudes of Movie 
Fans. New York: The Macmillan Co., 1933. Pp. 142. 

Ruth C. Peterson and L. L. Thurstone, Motion Pictures and the Social Attitudes of 
Children. New York: The Macmillan Co., 1933. Pp. 75- 

57 Richard S. Uhrbrock, “Attitudes of 4,430 Employees,” Journal of Social Psychology, 
V (August, 1934), 365-77- 
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prove suggestive for applications in the field of educational 
administration. 

The measurement of attitudes as a part of regular school 
work is given a brief but pointed discussion in the measurement 
volume of the Social Studies Investigation.** For immediately 
practical educational purposes, use for classroom appraisal is 
the ultimate goal of such measurement. Not until personality, 
in its many different phases, can be appraised by the teacher 
with as much significance to her as measurements of academic 
achievement have, and also with reasonable accuracy, can re- 
search workers or practical educators feel that appraisal pro- 
cedures have been brought to a satisfactory level. 

Further treatment of procedures which are appropriate for 
the study of personality will be found in later sections which 
deal with questionnaires, interview, observation, and appraisal. 
Because this field must be approached from so many different 
angles, requiring a variety of techniques, it cannot be treated 
as a unit in any one place in a text which is organized around 
methods and techniques rather than around fields of applica- 
tion. The student who is interested in working in this area will, 
therefore, consult the appropriate sections of this and the next 
chapter. 

Interpreting the results of survey testing. Before draw- 
ing conclusions from test results, one must consider the possible 
influence of a number of factors. The first question concerns 
the tests themselves. To what extent do they measure what 
they are supposed to, and to what extent do they measure 
something else? A careful analysis of any test will make clear 
that there are many factors which enter into each response, 
and the score consequently reflects many things in addition to 
what one desires. Sometimes these extraneous things become 
so numerous in proportion to the desired factors that the re- 
sults are practically meaningless. This is a consideration that 

58 Truman L. Kelley and A. C. Krey, Tests and Measurements in the Social Sciences. 
Report of the Commission on the Social Studies, Part IV. New York: Charles Scribner's 


Sons, 1934. Pp. 635. See especially pp. 75-95, and Chapter V, “Interests and Attitudes,” 
Pp. 340-471. ` 
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is more practical than theoretical, though it can be readily 
overlooked because it is not immediately obvious. Further dis- 
cussion of this danger is given in the next section, on construct- 
ing tests. 

In addition to spurious factors in the response of the 
individual that the test may call forth, there are various other 
chances for error or untrustworthiness in the results." The 
test may not sample a particular area of responses thoroughly 
enough so that the reactions secured give a proper picture of 
that area in the individual child. The test may have been 
administered poorly, or with varying directions and time limits 
for different groups of children. So few children may be sam- 
pled out of a school or school system that the results are not 
typical of the entire group in the school or school system. 

There may be still more serious discrepancies between what 
the results stand for and what one thinks they stand for. 
Various factors of a relatively fundamental nature may have 
entered into the training of the child, which are not at first 
suspected. Monroe estimates that children who are experienced 
in taking tests may do 10 per cent better than children who are 
not. He also reports an instance in which the average mental 
age of a group of children showed a gain of four and one-half 
years within a period of six months, because they were receiv- 
ing a special type of training that markedly affected the re- 
sponses which the particular test called for. 

In the interpretation of test results the assumption is com- 
monly made, that the curriculum is the same in all of the 
schools or classes which have been tested. One makes this 
assumption when he uses test results to draw conclusions con- 
cerning the quality of the teaching, or the diligence of the 
pupils, This is relatively clear ; but it is also true that this same 
assumption is made when the work in the different groups is 


59A brief, non-technical discussion of certain errors in measurement is given by 
Florence Goodenough and John E. Anderson, Experimental Child Study, pp. 406-11. 


New York: The Century Co., 1931. Pp. 546. 3 
60 Walter S. Monroe, “Dependability and Value of Survey Types of Investigations,” 


School and Society, XXXVIII (October 21, 1933), 517-22. 
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compared in any way, together with a suggestion as to the 
cause of its variation. In speaking of variation observed in 
the work of schools throughout a state, Monroe comments 
as follows: 


A large proportion of the variability of school averages in a state- 
wide survey of pupil achievement is doubtless due to differences in the 
objectives towards which the instruction in the several schools is di- 
rected. It may well be that, when the objectives are considered, a 
school occupying a relatively low position in the distribution of aver- 
age scores represents a higher degree of instructional efficiency than 
one occupying a relatively higher position.*t 


When one compares test results with a norm, other con- 
siderations enter. The norm is an average; it reflects all the 
factors which affected the performance of children who were 
used in establishing this norm. The first question one should 
raise therefore is, “Are the factors which have influenced the 
work of the pupils tested comparable to the factors which 
conditioned the work of the pupils by whom the norm was 
determined?” Many reasons exist in actual situations which 
invalidate such comparisons, and one will therefore be cau- 
tious in interpreting his objective findings. 

Ambitious teachers and principals sometimes raise the ques- 
tion, “Should one be satisfied just to have the average score of 
his class equal the norm?” The practical answer is that one 
should examine all of the factors conditioning the work of 
his pupils before he concludes that he is justified in making 
a comparison of any kind between their work and the norm. 
Perhaps his pupils should be well above the norm; perhaps 
they should be well below the norm. Such decisions cannot be 
reached merely on the basis of surface implications. In draw- 
ing conclusions from data, one will endeavor to keep in mind 
the possible influence of factors which he has not directly 
measured ; they may be more important than the factor he has 
studied. 

61 Ibid., p. 519. 


62 E. F. Lindquist, “Factors Determining Reliability of Test Norms,” Journal of Edu- 
cational Psychology, XXI (October, 1930), 512-20. 
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Test construttion. In connection with the discussion of sur- 
vey testing, a few comments will be made concerning the 
construction of tests. A similar section will follow each of 
the other types of normative-survey procedure. It should 
be made clear that these brief statements are only introduc- 
tions, or “overviews,” which aim to call attention to the prin- 
cipal considerations, rather than to go into complete detail. 
The technology of each specialized field is much too extensive 
to be included between the covers of one book and must be 
left for specialized treatises, of which there are many. The 
student will, therefore, be given definite reference to such 
treatises after a few paragraphs of general orientation which 
open up the subject. 

The prime consideration in the construction and adminis- 
tration of tests is validity—that is, representation of the influ- 
ence of factors that the test is supposed to reflect. Other 
considerations are largely of the administrative sort—the test 
must not require too long a time to give; it should be reason- 
ably easy to administer, etc. A test that is highly valid (for 
a particular purpose) will represent variation (from individual 
to individual or from time to time) in the characters it is sup- 
posed to measure with great fidelity. A test which has a low 
degree of validity secures responses, or an evaluation of re- 
sponses, which represent strongly the influence of a number 
of other factors, so that the character one desires to measure is 
somewhat lost among the many present. 

One does not need a full understanding of psychology to 
recognize that many factors enter into every response of the 
human being; that is, one responds to many stimuli at once, 
and his response is “colored” by a host of associations in his 
nervous system, of which he may not be at all aware. To get a 
response that represents a “pure” reaction of a certain kind, 
assuredly uninfluenced by any contributing or conditioning 
factors, is practically an impossibility and may be set down as 
a theoretical concept which does not in actuality exist. What 
we do practically, therefore, is attempt to secure test items 


. 
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which will stimulate responses of a given kind, and further, 
we try to get enough of these items of varying kinds so that the 
undesired factors will tend to neutralize each other and to 
“average out.” Investigators have so far attained only moderate 
success in these efforts. 

* One may ask at once whether a test of knowledge of arith- 
metic is not a measure of this ability and nothing else. Wherein 
lies the difficulty of constructing such a test? Such a position, 
though common, can only grow out of an inadequate under- 
standing of the complexity of human response. It will be 
immediately obvious that reading abilities, writing abilities, 
direction of attention, freedom from annoyances, and a normal 
supply of energy are required in responding to a simple arith- 
metic test of the usual kind. A person may fail to make his 
accustomed response because of a slight variation in attention, 
a slight cessation or diffusion of nervous energy, a slight varia- 
tion in the form of the stimuli, a slight (perhaps chance) 
variation in the pathway followed by the nervous discharge, 
and other things. It is well known that a child may respond cor- 
rectly with the sum of 5 and 3—to use a very narrow form of 
reaction as an illustration—under one set of circumstances and 
a few minutes later respond incorrectly when the circumstances 
are changed only moderately. Probably all of us have had the 
experience at times of unwittingly misspelling words that we 
knew perfectly well—or of not being able to think of the name 
of some person we knew well. 

Such variations in response exist in all degrees and represent 
the intrusion of various undesired factors into the response 
which is secured. When one considers the complexity of human 
response, he marvels that we can be trained to respond as 
consistently as we do. It is the goal of tests to be so constructed 
and administered and evaluated that few spurious (undesired) 
factors will enter at any point along the line. 

There are several methods of selecting test items and of 
validating them—i.e., checking them to see the extent to which 
they reflect the desired factors. One method is to select the 
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subject-matter (items) of the test as carefully as possible, 
taking precautions to see that only apparently crucial material 
is included, and that the items are presented in as clear a form 
as possible. Another method—perhaps the opposite extreme—is 
to select test material with little thought and then evaluate the 
test as a whole, and each item individually, by statistical 
methods. This evaluation may be made on the basis of several 
techniques which are treated in measurement books and in 
statistics books. 

Suffice it to say here, that neither of the above approaches 
is to be regarded as satisfactory in itself; both should be used 
to supplement each other. Even the utmost care in the selec- 
tion of material and in the formulation of items will not 
guarantee a satisfactory test (although they are certainly im- 
portant elements), and the product must therefore be tried 
out by as many means of objective appraisal as possible. On 
the other hand, one cannot rely solely upon correlation because 
so many things can be found that have a high correlation 
which do not have any immediate causal relationship—such, 
for example, as the increasing number of deaths from auto 
accidents and the increasing proportion of people who live to 
be over forty. One ordinarily desires test material that has a 
reasonably close connection with what he is trying to measure, 
on account of its effects upon the children and the teachers. 

With these few comments, the reader will be directed to lead- 
ing treatises on test construction, where he will find more de- 
tailed discussion. Most of this literature is technical and cannot 
be fully understood without training in statistics. One can, 
however, find sections of the material which are adapted to his 
understanding, whatever training he may have. 

A very readable chapter will be found in Smith and Wright’s 
book.® A rather brief, but helpful, treatment is given by 


63 See “Educational Tests and Their Uses,” pp. 482-3, 493-4, Review of Educational 


Research, V (December, 1935), No. 5. 
64 Henry Lester Smith and Wendell William Wright, Tests and Measurements, Chapter 


XXII, “Construction of Achievement Tests,” pp. 511-40. New York: Silver, Burdett 
and Co., 1928. Pp. 540. * 


322 EDUCATIONAL RESEARCH 


Tiegs.°° One of the older treatises, but one that contains much 
sound advice, is McCall’s, which also includes a brief discus- 
sion of making intelligence tests. Odell gives considerable space 
to the construction of essay and various kinds of objective 
tests in his older book on measurement.’ A widely accepted 
treatise on test construction, with an avowed emphasis upon 
objective tests, is that of Ruch.®* This re.erence also contains 
an excellent bibliography. A somewhat distinct note has been 
introduced into the measurement field by Tyler. His articles 
on test construction appear mainly in the Educational Research 
Bulletin (Ohio State University), beginning in 1930. Most of 
these can be obtained in a reprinted bound collection called 
Constructing Achievement Tests.® In more than a mechanical 
fashion, his emphasis upon validity is wholesome. Wright has 
prepared a very thorough discussion of the construction of an 
accomplishment-test battery for the State of Indiana.*® The 
principles underlying measurement have been given by Mon- 
roe.” The construction of tests especially for the social studies 
is treated by Kelley and Krey.’? 

The Review of Educational Research on “Educational Tests 
and Their Uses”? contains very ‘..portant discussions of 
this field and summarizes recent literature running into hun- 
dreds of references. The construction of educational and other 


65 Ernest W. Tiegs, Tests and Measurements for Teachers, Chapter XIV, “The Con- 
struction of Informal Objective Tests,” pp. 254-69. Boston: Houghton Mifflin Co., 1931. 
Pp. 470. 

66 William A. McCall, How to Measure in Education, Part II, “How to Construct 
and Standardize Tests.” New York: The Macmillan Co., 1922. Pp. 416. 

67 C. W. Odell, Traditional Examinations and New-Type Tests. New York: The Cen- 
tury Co., 1928. Pp. 469. 

68 G. M. Ruch, The Objective or New-T'ype Examination. Chicago: Scott, Foresman 
and Co., 1929. Pp. 478. 

69 Ralph W. Tyler, Constructing Achievement Tests. Columbus, Ohio: Bureau of Edu- 
cational Research, Ohio State University, 1934. Pp. 102. 

70 Wendell William Wright, The Development and Use of a Composite Achievement 
Test. Bulletin of the School of Education, Indiana University, V (January, 1929), No. 3. 
Bloomington, Ind. Pp. 90. 

71 Walter S. Monroe, An Introduction to the Theory of Educational Measurements. 
Boston: Houghton Mifflin Co., 1923. Pp. 364. 

72 Truman L. Kelley and A. C. Krey, op. cit. (See footnote 58.) 

73 W, J. Osburn, “The Selection of Test Items,” and G. M. Ruch, “Recent Develop- 
ments in Statistical Procedures,” Chapters II and III, pp. 21-40, in “Educational Tests 
and Their Uses,” Review of Educational Research, III (February, 1933), No. 1. 
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tests is reviewed also in the Psychological Bulletin." For later 
periodical references, see “Tests and Scales,” the many cross 
references, and “Tests and Scales—Construction” in the 
Education Index. Also see the topic, “Tests and Testing,” in 
the annual Bibliography of Research Studies in Education 
of the United States Office of Education. 

One can learn a great deal from studying tests that have 
been carefully made. Samples of the regular commercial tests 
may be purchased from the test publishers, and an excellent 
set of teacher-made tests is provided in a book by Ruch and 
Rice.”* 

There is practically no literature which is helpful in the 
construction of the essay test. In recent years this common 
and still useful instrument has been eclipsed by the emergence 
of the objective test; undoubtedly a large factor in this change 
has also been the fact that the objective test is much easier to 
study. There have been any number of investigations com- 
paring the essay and objective tests,”* but little that will help 
one make better tests of the essay or “old” type. Most of the 
suggestions offered are in the direction of trying to make the 
essay-type test take on as many of the characteristics of 
the new-type tests as possible. This attitude, carried out in 
practice, may result in a serious loss. The unique contributions 
of the essay test should be identified and preserved. 

Probably the reader will not undertake to construct an 
intelligence test. He can, however, locate treatises on its con- 
struction in books by Terman,” Pintner,’* and Freeman,” as 


74 For example, see the Psychological Bulletin, XXXII (July, 1935), 487-90. This 
series of annual reviews began in 1926. See footnote 48 for details. 

75G. M. Ruch and G. A. Rice, Specimen Objective Examinations. Chicago: Scott, 
Foresman and Co., 1930. Pp. 324. ` 

76 Charles C. Weidemann, “Recent Developments in the Written Essay Examination,” 
Chapter IV, pp. 484-90, in “Educational Tests and Their Uses,” Review of Educational 
Research, V (December, 1935), No. 5. 

77 Lewis M. Terman, The Measurement of Intelligence, Chapters III-IV, pp. 36-64. 
Boston: Houghton Mifflin Co., 1916. Pp. 362. 

78 Rudolph Pintner, Intelligence Testing, Chapter V, “Criteria and Construction of 
ee ae: Tests,” pp. 103-32. New York: Henry Holt and Co. (New edition), 1931. 
P. 555. 

79Frank N. Freeman, Mental Tests, Chapters IX and X, pp. 226-62. Boston: 
Houghton Mifflin Co., 1926. Pp. 503- 
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well as in periodical literature. Consult the usual indexes and 
also the Review of Educational Research® 

For the construction of personality tests, it is recommended 
that one study the existing instruments and review the discus- 
sions and investigations of these tests. The principal ref- 
erences to the literature have already been given in the section 
where personality tests were discussed. One may find especial 
interest in May’s general treatise on the problems of measure- 
ment in this field,* and in Hendrickson’s review of the as- 
sumptions involved.*? See also the references in footnotes 109- 
117 of Chapter VIII. 

Bixler has prepared a check-list 8 covering various aspects 
of research. He mentions ninety-six items that should be given 
specific attention in the making and use of tests. The student 
who is contemplating constructing or using tests in his research 
would do well to consult this list. 


2. STUDIES EMPLOYING QUESTIONNAIRE PROCEDURES 


The second type of normative-survey technique to be dis- 
cussed is the questionnaire inquiry. A questionnaire is a form 
which is prepared and distributed for the purpose of securing 
responses to certain questions, Generally these questions are 
factual, designed to secure information about conditions or 
practices of which the recipient is presumed to have knowl- 
edge. The questionnaire may, however, ask for opinions, and 
it may be used to afford an insight into the attitudes of a group. 
In fact there is no sharp dividing line between a questionnaire 


80 Harry J, Baker, “The Construction and Statistical Interpretation of Psychological 
Tests,” Chapter IV, pp. 295-9, in “Tests of Intelligence and Aptitude,” Review of 
Educational Research, IL (October, 1932), No. 4 
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82 Gordon Hendrickson, “Some Assumptions Involved in Personality 
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and a test, though they differ significantly in their common 
forms, 

The questionnaire is an important instrument in normative- 
survey research, being used to gather information from widely 
seattered sources. It is probably outranked in frequency of 
use only by the survey test. In fact, out of 581 printed studies 
representing research of all kinds, Koos found that in practi- 
cally one-fourth of them the questionnaire was used.™* A state- 
ment in the American School Board Journal sometime ago 
suggests that “the average school board in the city of 25,000 
population receives at least one questionnaire each day of the 
school year,” 5 If one considers all of the immediately practical 
studies which are made in educational field work (the opera- 
tion of schools), it is reasonable to believe that testing and the 
use of questionnaires would together account for over half of 
the studies that are made. There is in these statements no 
thought of implying that all or most of these immediately prac- 
tical studies are to be classed as research; but in normative- 
survey studies, whether of research character or not, tests and 
questionnaires rank high in frequency of use. 

The questionnaire procedure normally comes into use where 
one cannot readily see personally all of the people from whom 
he desires responses or where there is no particular reason to 
see them personally. The questionnaire may be used over any 
range of territory. International questionnaires have been 
used,®* In national studies questionnaires frequently occupy an 
important place. For example, in the recent National Survey 
of Secondary Education ®© extensive questionnaires were used 
to secure information about many different aspects of high 
schools, 

84 Leonard V. Koos, The Questionnaire in Education, pp. 6-13. New York: The Mac- 
millan Co., 1928. Pp. 178. 

85 “Questionnaires and Questionnaires,” American School Board Journal, LXXI 
(August, 1925), 74. 

86 See for example, Part-Time Schools: A Survey of Experience in the United States 
and Foreign Countries, with Recommendations. Bulletin, No. 73, Trade and Industrial 
ae 22, Washington, D. C.: Federal Board for Vocational Education, April, 1922. 


87 The National Survey of Secondary Education. Monographs Nos. 1-28, U. S. Office of 
Education Bulletin, 1932, No. 17. Washington, 1933-34. 
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Status studies. One of the favorite types of questionnaire 
study has been the determination of the status, including per- 
sonal and professional characteristics, of various school officials 
and teachers. Such studies have been made on a national basis 
for the high-school principal,’* the junior high-school princi- 
pal,” the city superintendent of schools,’ the business 
manager,” and rural school teachers, principals,” and superin- 
tendents.” Similar studies on a state-wide basis have been 
made in many states * and are favorite subjects for Masters’ 
theses.” For other analyses of the status of various groups, see 
the topic, “Analyses of the Activities and Characteristics of 
Educators, Scientists, and Psychologists,” in Chapter III, some 
of which are based on questionnaire data. 

Questionnaires are rather commonly used even within single 
cities to secure information from the teaching staff about such 
problems as training, length of service, duties, etc., when a city 
school system is being surveyed. In fact, a superintendent of 
a large city school system might circulate an informal ques- 
tionnaire among the members of the central office staff in order 
to find out certain things from them—for example, the number 


88 Dan Harrison Eikenberry, Status of the High-School Principal. U. S, Bureau of 
Education Bulletin, 1925, No. 24. Washington, 1926. Pp. 71. 

89 Frank Kale Foster, Status of the Junior High-School Principal. U. S. Office of Edu- 
cation Bulletin, 1930, No. 18. Washington, 1930. Pp. 75. 

90 Educational Leadership, Chapter VI, “The Progress and Status of the City Superin- 
tendent of Schools,” pp, 99-149. Eleventh Yearbook of the Department of Superintend- 
ence. Washington, D. C.: National Education Association, 1933. Pp. 528. Bibliography. 
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First Yearbook of the Department. 

91 Amos L, Heer, The Present Status of Bu 
the United States in Cities of 25,000 and M 
(Director of Teacher Training, Kent State Ni 

92 Walter H, Gaumnitz, Status of Teacher 
Schools of the United States, U. S, Office of 
ton, 1932. Pp, 122. 

93 “The Status of the Superintendent of Rural Schools,” 
Educational Leadership, (See tootnote 90.)' Bibliography. 

94 Orlie M. Clem and James F, Murray, “The Status of the Pennsylvania High- 
School Principal,” Educational Administration and Supervision, XIX (September, 1933), 
442-50. 

Omer Hibben Bennett, “The Status of County Superintendence in Ohio.” Master’s 
thesis, University of Cincinnati, 1929, Pp. 44. (Also published for private circulation.) 

95 See such topics as “Teachers—Status,” ‘Administration of Schools: Supervision 
and Supervisors,” “School Administration: School Principals,” etc., in the annual 
Bibliography of Research Studies in Education, published by the U. S. Office of Educa- 
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who wanted to be absent from the office to attend the Depart- 
ment of Superintendence Meeting of the National Education 
Association. 

Current practices. Questionnaire studies concerning current 
practices in school systems constitute a second group which 
is prominent. The Research Division of the National Education 
Association, which regularly prepares excellent questionnaires, 
has made a number of interesting questionnaire studies of cur- 
rent practices and current conditions.°° The American Library 
Association makes an annual questionnaire study of the prac- 
tices and conditions with reference to school libraries through- 
out the country; this is published in the Bulletin of the A. L. A. 
Reavis and Woellner * used the questionnaire to ascertain 
typical practices of high-school executives in equipping their 
offices, and in administering routine affairs. Replies were re- 
ceived from 522 principals. As a part of a national study of 
Boy Scout work, Wyland ® sent a questionnaire to schools 
in selected cities to learn the degree of school coöperation with 
Scout work, 

In the field of teacher personnel, “survey studies based on 
official records or questionnaire returns predominate.” ® These 
studies are concerned with current practices in the selection 
and placement of teachers, the teaching load (size of class, 
number of classes, number of subjects, etc.), the combinations 
of subjects taught, and the status of groups of teachers with 


96 “The School Board Member,” Research Bulletin of the National Education Associa- 
tion, XI (January, 1933), 1-41. 

“Practices Affecting Teacher Personnel,” Research Bulletin of the National Education 
Association, VI (September, 1928), 207-53. 

“The Nation’s School Building Needs,” Research Bulletin of the National Education 
Association, XIII (January, 1935), 3-34- 
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tion, No. 631. New York: Teachers College, Columbia University, 1934. Pp. 200. 
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regard to tenure, health, legal regulations, supply and demand, 
etc.’°° Reavis states that 


questionnaires and check-list inquiries, including the reports made to 
state departments of education and the United States Office of Educa- 
tion, have constituted the chief source of information regarding cur- 
rent practices in public-school organization and administration,1° 


Financial studies. Financial questions occur frequently in 
questionnaire studies. Financial problems have recently been 
pressing; indeed, they are always of large moment. The de- 
pression prompted all sorts of questionnaires, frequently circu- 
lated within a single state, to ascertain the tax rate that the 
local district levied for schools, the amount of extra levies, 
the amount of bonded indebtedness, the amount of current 
indebtedness, the average expenditure per pupil, the average 
teacher’s salary, how much the average salary had been re. 
duced, how much other expenditures had been reduced, etc. 
In this connection, the biennial salary questionnaires of the 
National Education Association are outstanding. These have 
been used for a number of years, and are published in the Re- 
search Bulletin of the N. E. A» The importance of these 
studies is perhaps indicated by a statement in the Fi oreword of 
the 1935 publication: “A steady stream of letters has been re- 
ceived calling for these findings at the earliest possible 
moment.” 

Many other fields and subjects have been studied by means 
of the questionnaire. Koos 1 mentions the following in order 
of frequency in his sample: administration, curriculum, tests 
and measurements, psychology, methods of teaching, and 

100 For illustrations of studies see “Teacher Personnel,” Review of Educational Re- 
search, IV (June, 1934), No. 3, and I (April, 1931), No. 2. 

401 William C. Reavis, “Methods of Research in School Organization,” p. 26, Chap, 
HI in “Methods and Technics of Educational Research,” Review of Educational Research, 
IV (February, 1934), No, 1. 

102 “Salaries of School Employees, 1934-38,” Research Bulletin of the National Edu- 
cation Association, XIII (March, 1935), p. 30. This is the seventh biennial survey of 
salaries. In addition to this bulletin, detailed distributions are issued in five separate wub. 
lications known as Special Salary Tabulations. These show distributions by the individus] 
city, and are priced at $3.00 for each publication. 
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supervision. For further examples one should look over issues 
of educational magazines which report studies, or lists of Mas- 
ters’ and Doctors’ theses (see Chapter III). The National 
Education Association issues an annual bibliography ** of 
questionnaire studies completed and filed in their office by 
those who are codperating in their “questionnaire service.” 
A Master’s thesis at the University of Colorado analyzes 500 
questionnaire studies. One should also consult “Question- 
naires” in the Education Index for current discussion of them, 

Many service studies do not find their way into print, espe- 
cially since mimeographing is so prevalent and can frequently 
supply the needs for summaries of state-wide studies. For 
example, a recent study by Holy of the practices of Ohio 
schools with reference to carrying fire insurance *°* was issued 
in mimeographed form. 

Check-lists. Questionnaires sometimes take the form of a 
check-list, which is simply a set of categories for the respondent 
to check. For example, Ayer #7 submitted a check-list contain- 
ing 1,000 duties to principals and superintendents, asking them 
to check those duties which they performed and to indicate 
roughly the frequency (such as daily, weekly, etc.). Wick- 
man% used a check-list with teachers to ascertain the fre- 
quency with which certain undesirable acts of pupil behavior 
occurred, Charters ?° submitted a check-list of duties to 2,000 
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secretaries to learn the frequency with which different duties 
were performed. In tne Commonwealth Teacher-Training 
Study,’ Charters and Waples distributed elaborate lists of 
duties to teachers all over the country to ascertain the fre- 
quency of performance of each activity. 

Check-lists are also used for other purposes, which will be 
discussed later. For example, they may be employed as a form 
of recording when one is observing behavior, and they may 
also be used as an instrument for rating schools, textbooks, etc. 
These uses will be discussed in the following chapter under 
“Observation and Appraisal Procedures.” 

Studies of attitudes. Questionnaires are not necessarily con- 
fined to statistical data, or even to factual material. They may 
enter the field of attitudes, opinions, and judgments. It is in 
these areas that great care must be observed. One must be 
careful in preparing his questionnaire and in interpreting his 
returns, to recognize opinion as such. There is a legitimate field 
for the questionnaire in getting a cross section of thought. In 
this form the questionnaire may approach the test form so 
closely that one cannot draw a definite line between them and 
may not know whether to call his instrument a test or a ques- 
tionnaire. There is a possibility that it is both.” 

Two recent Doctors’ theses may be mentioned in this con- 
nection. Ward reported on the Philosophies of Administration 
Current in the Deanship of the Liberal Arts College“? In mak- 
ing this study the author sent out three different question- 
naires. The first one secured information on age, sex, race, 


letin No. x. New York: The National Junior Personnel Service, Inc., 70 Fifth Ave., 
1924. Pp. 62. 

110 W. W. Charters and Douglas Waples, Commonwealth Teacher-Training Study, pp. 
19-27, 77-114, and 254-303. Chicago: University of Chicago Press, 1929, Pp. 666. 

111 See for example the questionnaire surveys of personality and adjustment referred 
to by Goodwin B. Watson, in “Tests of Personality and Character,” especially pp. 
186-93, Review of Educational Research, II (June, 1932), No. 3. 
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“The Questionnaire,” Part II, pp. 122-50. New York: Harper and Bros., 1933. 

112 Merle Scott Ward, Teachers College Contributions to Education, No. 632. New 
York: Teachers College, Columbia University, 1934. Pp. 128, 


THE NORMATIVE-SURVEY METHOD 33) 


academic degrees held, honorary degrees held, foreign travel, 
foreign study, previous subject-matter field, kind of previous 
professional experience, tenure, and length of total educational 
experience, for the college deans. This questionnaire dealt with 
data of the more usual type (factual). The second question- 
naire dealt with the attitude of the deans toward various prob- 
lems in four areas of college administration—the purpose of a 
liberal arts college, the curriculum, the improvement of instruc- 
tion, and student welfare. This questionnaire was a direct effort 
to sample the opinions of the deans ; it was in the form of mul- 
tiple-choice statements, the alternatives representing various 
possible attitudes toward each question that was asked. The 
questions were so prepared that they fitted into a pattern, and 
the writer could abstract from them certain large generaliza- 
tions regarding the philosophic pattern of the individual re- 
sponding. As an additional line of evidence, the author 
distributed Donald P. Cottrell’s “Test on Controversial Issues 
in Higher Education” (revised edition)."* Like the second 
questionnaire, this elicits opinions that enable one to draw 
certain conclusions about the general attitude of the respond- 
ent. Three hundred and thirty college deans responded to the 
entire set of questionnaires. 

Another illustration of this general class will be given. Bair 
studied The Social Understandings of the Superintendent of 
Schools.1‘+ He likewise used three forms. The first gathered 
information about the racial and social origins of the super- 
intendent, the conditions of his childhood, his education, ex- 
perience, and social and professional relations with the 
community. The second form contained thirteen questions. 
These are interesting because of the type of response they 
called for, For example: (1) Will you indicate in a word or 
brief sentence three of the most fundamental questions in your 
opinion facing people in the United States to-day? ( 3) Do you 

113 First described in “Measurement of Conflicting Viewpoints in Higher Education,” 
Teachers College Record, XXXIV (May, 1935), 635-54- 


114 Frederick Haig Bair, Teachers College Contributions to Education, No. 625. Nev 
York: Teachers College, Columbia University, 1934- Pp. 193- 


332 EDUCATIONAL RESEARCH 


think that the public schools ought to deal with such questions? 
(4) Do you consider that the social studies as taught in our 
public schools generally, now provide for a reasonably thorough 
study of these and similar critical questions? (7) Will you be 
good enough to... indicate groups likely to favor or to oppose 
the study? 

The difficulty of tabulating responses to such questions as 
the foregoing is apparent. Nevertheless, the author was dealing 
with a fundamental problem, and he sought what he wanted. 
In addition to these two forms, the author sent out Manly 
Harper’s “A Social Study,” which consists of 71 multiple- 
choice propositions designed to reveal the degree to which one 
is conservative in his social attitudes. These three question- 
naires elicited responses from about 850 superintendents—one- 
fifth of the number of inquiries sent out. ns per cent of 
replies is lower than usual. 

A distinction must be made between the type of subjective 
response represented in these two studies and that which is to 
be interpreted as an objective fact. The opinions and attitudes 
represented are facts in so far as the responses are typical re- 
sponses of the individuals, but they are facts of opinion. They 
represent the leanings of the group—tendencies which may be 
either right or wrong, helpful or detrimental to society, gener- 
ally good or generally bad. These facts of opinion are different 
from opinions about facts, which are normally untrustworthy. 
If, for example, one asks all the superintendents in the country, 
“What was the main cause of the 1929 depression?” the average 
opinion in the replies might be no nearer the truth than one’s 
own. As the National Education Association Research Division 
says: 71° 


The hastily expressed opinions of a large number of judges of vary- 
ing merit are probably of less value than the mature judgment of one 
or two experts who have the question clearly in mind, and who are 
able to qualify their answers appropriately. 
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Other aspects of personality besides opinions or attitudes 
are studied by means of questionnaires. Symonds "° devotes 
a chapter each to the adjustment questionnaire, the attitude 
questionnaire, and the interest questionnaire. It is apparent 
that the questionnaire has an important rôle in this field as well 
as in the realm of securing objective facts. 

Consideration for the respondent. The essentially co- 
operative nature of the questionnaire must not be over- 
looked. Probably most of the abuses of the instrument lie 
in this connection. Students and others are so close to their 
own study that they lose perspective concerning what it is 
reasonable to ask of another person—usually a complete 
stranger. One can be reasonably certain that his question- 
naire goes to people who are already busy, and although they 
would like to help every person who is making a study, they 
have large obligations for the direction and prosecution of 
enterprises with which their positions are charged. One may 
also be reasonably sure that the regular duties of the person 
to whom he sends his questionnaire are at least equal in 
importance to the study which is being undertaken. This 
has two practical implications: first, do not undertake a 
questionnaire study unless the problem is really an impor- 
tant one—not to you as an individual, but to education ; 
second, devise your questionnaire so that it will not make 
any more demands on the time of the respondent than 
necessary. 

Relation to reports. It may be interesting to make a brief 
comparison of questionnaires and reports. Both are nor- 
mally means of securing information from people at a dis- 
tance. Reports, however, are typically required by some one 
with authority, and there may be penalties for not making 
them, for omitting some item of information, or for not 


116 Symonds, op. cit. (footnote 111), Chapters V-VII. See also Garrett and Schneck, 
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completing them by a certain date. Again, reports are usu- 
ally periodical. This means that the procedures for securing 
the information called for are likely to be established and 
routine; at least, preparation can be made in advance to 
meet the situation of filling out the report at a given time. 
In the third place, reports usually deal largely with figures; 
they do not cover the variety of responses which are fre- 
quently contained in questionnaires. In the fourth place— 
and this concerns the use and interpretation of the results— 
because of the conditions mentioned the returns are likely 
to be from the entire group in the case of the reports, so 
that the sampling problem is not present to cast doubt upon 
one’s interpretations. 

Reports can often be used to supplement, or take the 
place of, certain types of questionnaire. It must be recog- 
nized on the other hand that the versatility of the ques- 
tionnaire and the freshness of its returns—reports are 
somewhat slow in being assembled and summarized—tend 
to make it an indispensable instrument for current infor- 
mation and for research. Those who expect to employ this 
instrument successfully, however, must guard against the 
gross lack of judgment which not infrequently characterizes 
the preparation and use of the questionnaire. 

Interpreting questionnaire returns. The questionnaire 
technique provides certain opportunities for special inter- 
pretation, as well as special difficulties. At the beginning of 
this chapter, the authors emphasized the fact that normative- 
survey studies should be valuable in broadening one’s per- 
spective and should serve to call attention to desirable ways 
of solving problems. The results should throw light on emer- 
gent practices and upon novel or unique situations. If such 
possibilities are to be realized, the returns must be analyzed 
carefully. The student must examine individual situations 
as such, seeking for elements of interest and significance in 
each. If he depends upon a mechanical (clerical) tabulation 
of questionnaire results concerning practices and conditions, 
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he will miss many of the opportunities for real insight into 
conditions. He will probably desire to secure the statistical 
aspects of his returns, but he will also bear in mind that 
figures tell but a small part of any story, and he will be 
alert for discovering other valuable aspects of data. 

The interpretation of frequency studies solely in terms 
of the figures is not only inadequate but difficult. For ex- 
ample, if 75 per cent of the superintendents who reply to a 
questionnaire indicate that they solve a problem in a certain 
way, this may be as much as one needs to know for pure 
science, but for practical purposes one must go farther and 
attach a certain significance to this fact. What shall the 
conclusion be? Does this fact mean that this is a desirable 
way to solve the problem? Is it necessarily more desirable 
than another method that comes to light in the survey, of 
which the 75 per cent had not yet thought, or heard? Or, if 
60 per cent of the principals indicate on a check-list that 
they perform a certain duty, on the average, less than once 
a month, can one tell whether the duty is important or not? 
Can cruciality be inferred from frequency? Or can the im- 
portance of an event for a single individual be inferred from 
the prevalence of the event in the whole group? Finally, will 
frequency of performance tell whether an activity should be 
taught in a training school or left to one’s general culture 
and common sense? 

It was in recognition of such problems that Charters and 
Waples "° secured indices on four different aspects of the 
activities contained in their master-list of teachers’ activities. 
From teachers they secured the frequency with which ac- 
tivities were performed, and from teachers and others they 
secured ratings on the desirability of these activities, on the 
difficulty of learning to perform the activities, and on the 
appropriateness of learning the skill in a teacher-training 
institution. With these four criteria, one is in a much better 
position to secure some guidance from the list than he would 
be if he had only frequency. Further discussion of this gen- 
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eral problem, and of the Charters-Waples study in particular, 
will be given in connection with the interpretation of docu- 
mentary frequency studies and appraisal procedures. 

In addition to counting, the full interpretation of question- 
naire returns definitely involves evaluation. The exercise of 
judgment is necessary in deciding what practices are worthy 
of emulating. It is necessary in drawing practical conclusions 
of any kind from the data. We may illustrate this further 
by referring to another use that is sometimes made of such 
data. It is commonly assumed that if the conditions in one’s 
own school system, or classroom, are about the same as the 
conditions revealed by the majority of the returns, one may 
be complacent. It is sometimes stated that the middle 50 
per cent of the cases represent a safe zone—that is, between 
the limits which cut off the upper and the lower fourths of 
the group representing the more extreme divergencies from 
average or majority practice. One may feel comfortable, yes, 
and such facts are very telling in meeting public criticism 
of the schools; yet one knows that progress comes only 
through the practices of those who do diverge from the 
mode. The significance of the findings concerning majority 
practice will, therefore, vary widely just as one’s philosophy 
varies from that of wanting his school system to be in the 
“safe” class or among those systems which are striving more 
actively to adapt themselves to changing conditions and 
needs. 

Another source of difficulty arises from failure to respond 
to one’s request for information. Questionnaires which are 
not returned and incompleted questionnaires present difficult 
problems of interpretation, particularly when the returns 
are expressed in per cent. One has always to consider the 
question, “If all of the blanks had been returned, would 
they have shown a significantly different picture?” This 
question is not a problem of simple sampling, for there 
may be a definite relationship between the response or lack 
of response and the existing conditions. Perhaps the recipient 
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did not respond because conditions were very good, and he 
saw no point to the study. Or, conditions may have been so 
unsatisfactory that he did not care to reveal them. Or, data 
may have been inaccessible to him, or his time too full. What 
is the probable reason for the absence of the response? The 
research worker must consider these possibilities before 
drawing his conclusions. The problem is indeed baffling and 
lies entirely outside of any statistical technique of allowing 
for sampling fluctuations or “unreliability. At best the inter- 
pretation is made with certain hazards.” 

Consideration of biased or inaccurate responses, of ambigu- 
ous statements, of statements which do not furnish logical bases 
for the inferences drawn, and other technical questions do not 
fall within the scope of this treatise. 

Questionnaire construction. In thinking of a questionnaire 
study, perhaps the first questions one should ask himself are, 
“Why would these people answer the questions I am framing to 
send them?” “Is there a good reason, from their point of view, 
for taking the trouble to give answers?” We have said that 
questionnaires are requests for information, usually sent to 
strangers who have their time more than filled. When framing 
his questionnaire, one will therefore bear in mind the demands 
which he is making upon another person’s time. 

This can be done in several ways. In the first place, one can 
scrupulously weed out every trivial question ; in some cases the 
number of questions can be reduced by one-half. In the second 
place, one can make the responses simple—possibly involving 
only check marks after a variety of suggested answers. In the 
third place, one should study his questions assiduously to see 
that no unnecessary specifications or details are included in 
them, One can make a question much more difficult to answer 
by thoughtlessly specifying certain things. For example, if one 
wishes only to know how much the average of all teachers’ sal- 
aries in a city has increased or decreased from one year to 


117 Florence L. Goodenough and John E. Anderson, Experimental Child Study, “The 
Reliability of Questionnaire Data,” pp. 384-90. New York: The Century Co., 1931+ 
Pp. 546. 
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another, it may mean considerably less work for the respondent 
if this question is asked directly instead of requesting the 
average salaries for each of the years in question. In the fourth 
place, one should not ask for information which is obtainable 
from documentary sources available to the sender. 

Another requirement for questions—important from the 
standpoint of the respondent—is that they apply to his situ- 
ation. Often the maker limits his question unnecessarily and 
thus fails to secure a desired response, because the query as 
stated does not fit the conditions obtaining where it is sent. Dis- 
crepancies of this sort frequently occur when questionnaires 
are prepared for cities of a certain general size and are sent out 
to cities of a very different size. Not only do such ill-adapted 
questions often fail to secure a response that the maker desires, 
but they irritate the one filling out the questionnaire so that 
he may give hasty answers or fail to return the document at all. 
One must think of the psychology of the respondent. 

Of course, many other considerations enter into the making 
of a questionnaire from the standpoint of the study which is 
being made. First, one must have a clear purpose, with definite 
limitations, so that he does not ask for everything in “blunder- 
buss” fashion: he must see how each item of information fits 
into a pattern of essential knowledge about his problem. Second, 
each question must be absolutely clear—not only to the maker 
but to the receiver. It is surprising how many questionnaires are 
sent out that are scarcely interpretable. Third, one should seek 
responses of such character that they can be summarized in 
some form. This does not necessarily mean that the responses 
must be quantitative, or yes-no, or check marks, although these 
are the easiest to summarize. It means that the step of sum- 
marization is one which should be considered when the ques- 
tions are being prepared. Fourth, one will refrain from asking 
questions of opinion unless he is certain that opinion is what 
he is seeking, and that it will be worth getting. Fifth, one will 
consider the desirability of pre-coding his questionnaire. This 
ig frequently done when the results are to be punched on 
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tabulating machine cards for summarization. The problem of 
coding and of operating tabulating machines is a separate study 
in itself; it is, however, essential in expeditiously handling a 
large number of returns.*** 

One should invariably get assistance from others in criticiz- 
ing his questionnaire before it is sent out finally. The least one 
can do is to request some of his friends to look over the items 
critically with the thought of discovering possible misinter- 
pretations. One should then send out a few copies of the ques- 
tionnaire to some members of the group and examine the 
returns from these before the questionnaire is used on a large 
scale. Tests, questionnaires, classification schemes, check-lists, 
rating scales, score cards, and other such schedules, all need 
validation in terms of practical use, in addition to whatever 
theoretical and statistical precautions may have been taken in 
their initial preparation. An experienced worker will not hope 
to prepare such an instrument and to have it perfected through 
his thinking alone; he has learned long since that no one person 
can think in all of the ways that a group can, and that accord- 
ingly he cannot anticipate adequately the interpretations of 
others and the varying complexities of situations which are 
foreign to him. He will, therefore, seek the critical reactions of 
many persons, as well as secure the desired responses from 
others, in a preliminary way, before trusting his schedule to 
actual use. 

The check-list form of questionnaire is particularly depend- 
ent upon completeness. Its purpose is to afford both a con- 
venient and a suggestive list on which recipients may check. 
The respondent comes to depend upon the list for suggestive- 
ness and for a classification of his responses, so that he is not 
so likely to write in additional items. In fact, items which he 
might intend to record, if there were no categories at all, may 
be omitted when a list that does not contain them is given him, 
either because he deems the given list to be inclusive of all 


118 See the discussion and references ov the tabulating machine technique given in 


Chapter XI. 
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that is desired, or because he assumes a mind-set of dependence 
on the list, 

Various means are available for discovering items to in- 
corporate inte the check-list, and to test it for completeness, 
These means will vary somewhat with the problem. Two illus- 
trations may be taken from Charters’ work. In the study of 
Secretarial traits,1°° r2 5 secretaries were interviewed to as- 
certain the various tasks which they performed, A list of 871 
tasks was thus secured—obviously more than any one person 
would think of. In the case of the Commonwealth Teacher- 
Training Study "° more elaborate precautions were taken. In 
preparing the check-list of teacher-activities for distribution 
to teachers to check against (for Securing frequencies), the first 
step was to study activity lists prepared previously. The search 
for these earlier investigations yielded twenty studies with an 
aggregate of 6,000 activities, During one summer several thou- 
sand teachers were asked to list their activities, after being given 
careful directions. Experienced teachers added other activities 
through conference, 

A classification was attempted at this point and worked out 
by means of a hierarchy consisting of divisions, subdivisions, 


of the activities submitted were classified under this scheme, 
after which the list was printed and submitted to over two 
thousand teachers to check for completeness. Books and ar- 
ticles were examined to see if new activities would be sug- 
gested; graduate students in twenty-five different summer 
classes wrote down activities they thought should be performed 
by teachers, and when no new activities appeared through these 
steps, the list was regarded as satisfactorily complete. 

The preparation of a list of professional and personal traits 
of teachers, as another part of this same Study, is perhaps even 
more unique. Two thousand eight hundred desirable “actions 
indicating the possession of traits” were secured by interview- 
ing administrators, teachers, pupils, and others, Obviously, 
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such a list was too large for practical use. The first step in 
reducing it was “translating” the actions into traits; that is, 
twenty-one judges were used to name the traits represented by 
each of these actions. This step classified the actions under 
eighty-three traits. The meanings of the different traits had 
been determined by using the definitions agreed upon by three 
out of four dictionaries. The various actions were assembled 
under each trait, and the eighty-three traits were then “tele- 
scoped” by judges into a final list of twenty-five; that is, the 
judges were told to combine those traits which were practically 
the same. The resulting list of twenty-five traits was considered 
of usable length and was subsequently distributed for evalua- 
tion? 

One should try tabulating or summarizing the data which 
he receives from a preliminary trial of his blank, before send- 
ing the questionnaire out in quantity. This attempt to handle 
the data may lead to the revising of certain questions. One 
should carry the process of analysis forward, as though these 
data constituted his complete returns. He should ascertain 
whether the data warrant conclusions which are significant 
for his purpose. Even if it should not be practicable to secure 
an advance sample of returns, one should, nevertheless, draw 
up tables which will provide for summarizing the information 
that he expects to receive. He should then study: these tables 
to see if they afford answers to his main questions, and to the 
subordinate questions which add interesting sidelights to his 
main study. If carefully done, such an analysis should aid in 
discovering any additional information that will be needed to 
round out the study properly. It may also serve to reveal cer- 
tain useless questions that are in the blank. 

Finally, one must not overlook the importance of selecting 
carefully the group to whom he sends his questionnaire. This 
selection involves a good reason for believing that the people 
receiving the questionnaire will be in a position to give the 
information desired; and, where all the members of any group 

119 Charters and Waples, op. cit. (footnote 110), pp. 14-7, 51-76, and 223-44. 
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do not receive questionnaires, the selection involves sampling 
problems. Further, if people have the information, or knowl- 
edge, are they free to respond? In some cases they are free 
but unwilling to do so; +° in that case, a form of questionnaire 
which has the name detachable may be desired.1*? One should 
be ready with some plans for following up his questionnaire 
in order to secure a larger per cent of returns, in case the 
response does not reach expectations.” 

There is an ample literature on the questionnaire, and the 
reader is referred to this for an adequate treatment of the 
many problems which arise. Probably Koos’ book ?*° should be 
consulted at the outset, especially Chapters III-VII. The 
National Education Association report on the questionnaire 1” 
should by all means be read. Goodenough and Anderson 2*° 
discuss seven points to be watched in constructing a ques- 
tionnaire. Symonds’ four chapters on the questionnaire +8 con- 
tain an excellent digest of studies and many references to 
other sources of information. It is assumed that one will 
acquaint himself with the references given in the preceding 
discussion of the uses of the questionnaire, since many of them 
have an important bearing on questionnaire construction. 
Bixler 1*7 gives forty-six points to check against in the prepara- 
tion and use of a questionnaire. 


120S, M. Stoké and H. C. Lehman, “The Influence of Self-Interest upon Question- 
naire Replies,” School and Society, XXXII (September 27, 1930), 435-8. 

121 See, for example, the form used in the study of Cleveland teachers’ salaries (pasted 
inside back cover), T. C. Holy and Others, Cleveland Teachers’ Salaries. Bureau of Edu- 
cational Research Monographs, No. 16. Columbus, Ohio: Ohio State University, 1932. Pp. 
196. See also the questionnaire used in the study by the National Educational Association 
Committee on the Economic Status of the Teacher, B. R. Buckingham, Chairman, pub- 
lished as “The Teacher’s Economic Status,” Research Bulletin of the National Education 
Association, XIII (September, 1935), No. 4. See p. 171. 

122 E. E. Lindsay, ‘Questionnaires and Follow-Up Letters,” Pedagogical Seminary, 
XXVIII (September, 1921), 303-7. 

Herbert A. Toops, “The Returns from Follow-Up Letters to Questionnaires,” Journal 
ofe Applied Psychology, X (March, 1926), 92-101. 

128 Leonard V. Koos, The Questionnaire in Education, New York: The Macmillan Co., 
1928. Pp. 178. Includes twenty-one references in the bibliography. 

124 “The Questionnaire,” Research Bulletin of the National Education Association, 
VIII (January, 1930), No. 1. Pp. 51- 

125 Goodenough and Anderson, op. cit. (footnote 117), “The Construction of a Ques- 
tionnaire,” pp. 390-402. 

126 Symonds, op. cit. (footnote 111), Chapters IV to VII, pp. 122-259. 

127 H, H. Bixler, Check-Lists for Educational Research, pp. 40-5. New York: Teachers 
College. Columbia University, 1928. Po. 118. 
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Several additional references are cited for such assistance 
as they may offer.** Others can be located by consulting the 
topic, “Questionnaires,” in the subject index of the annual 
Bibliography of Research Studies in Education published by 
the U. S. Office of Education and by referring to the same topic 
in The Education Index. 


3. STUDIES EMPLOYING DOCUMENTARY-FREQUENCY PROCEDURES 


This type of normative-survey research, like historical re- 
search, deals with records which already exist. The present 
type of research, however, is definitely quantitative; it is not 
concerned with the general import of the existing documents, 
but with certain characteristics which can be identified and 
counted. Determining what characteristics to count, and de- 
fining them, are important parts of this type of work; in fact, 
they may be the most crucial part. 

The fact that one works directly from documents does not 
mean that he avoids all problems of collecting and selecting 
data. It is true that in some cases he needs only to procure a 
few books from a library, but it is also true that in other cases 
he may need to collect his documentary specimens from afar. 
In any case, problems of selection enter, both with respect to 
the specimens which are collected, and with respect to the 
aspects of these specimens which are noted. In dealing with 


128 Ruth S. Cavan, “The Questionnaire in a Sociological Research Project,” American 
Journal of Sociology, XXXVIII (March, 1933), 721-7- 

W. H. Cowley, “Two Questionnaire Devices,” Educational Research Bulletin, X (Oc- 
tober 14, 1931), 374-6. 

Margaret M, Gaines, “The Questionnaire: Technique and Its Preparation and Use.” 
Master’s thesis. Pittsburgh: University of Pittsburgh, 1926. Pp. 242. 

Max Seham and Ole Schey, “The Reliability and Validity of the Questionnaire 
Method,” Research Quarterly of the ‘American Physical Education Association, V (May, 
1934), 31-43. 

F. x Shuttleworth, “A Study of Questionnaire Technique,” Journal of Educational 
Psychology, XXI (December, 1931), 652-8. 

Mapheus Smith, “A Note on Stability in Questionnaire Response,” American Journal 
of Sociology, XXXVIII (March, 1933), 713-20. a 

See references on “The Questionnaire,” pP- 93-96, in An Annotated Bibliography on 
the Methodology of Scientific Research as Applied to „Education, by A. S. Barr and 
Mabel Rudisill. Bulletin of the Bureau of Educational Research, No. 13. Madison: 
University of Wisconsin, June, 1931. Pp. 129. 


344 EDUCATIONAL RESEARCH 


such questions of selection, one must always remember the 
type of generalization he desires to make as a result of his 
study. 

Textbook analyses. Among the simpler examples of docu- 
mentary frequency studies may be mentioned textbook an- 
alyses. These have become a favorite subject for Masters’ 
theses. A student may choose any set of objective character- 
istics which he thinks are significant in a book and note the 
extent to which they occur. For example, he may count the 
number of sentences of different lengths, the number of words 
which appear to be diffcult,?° the number of pictures,"*° the 
number of tables, the number of exercises at the ends of the 
chapters, the number of different topics taken up, the number 
of lines (or pages) devoted to each topic, etc. Other investiga- 
tions may be concerned somewhat more broadly with the kind 
of content, perhaps in relation to the objectives in that 
subject,'* or to other criteria? Some studies deal with the 


129 Laura Margaret Elder, “A Determination of Vocabulary Grade-Placement of Certain 
Home Economics Textbooks” Master's thesis, Los Angeles: University of Southern Cali- 
pris, 1933. Pp. 65. See also, California Quarterly of Secondary Education, IX (October, 
1933), 83. 

Grayson Newark, “An Investigation of the Vocabulary Burden of Dull’s High- 
School Chemistry Text: Modern Chemistry.” Master's thesis. Ann Arbor, Mich.: Uni- 
versity of Michigan, 1933. Pp. 109. 

Frances Summerell, “Relative Difficulty of the Vocabulary of the Ten Most Frequently 
Used High-School Biology Textbooks.” Master’s thesis, Pittsburgh: University of Pitts- 
burgh, 1932. Pp. 171. Abstract printed in: Abstracts of Theses, Researches in Progress, and 
Bibliography of Publications, of the University of Pittsburgh, Vol. VIII, 373-4. 

130 Helen A. Creese, “An Evaluation and Classification of Pictures Used in Geography 
one from 1880 to 1930.” Master's thesis, Pittsburgh: University of Pittsburgh, 1933, 
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Nelda Ruth Pokel, “Evaluation of Graphic Representation in Junior High-School Text- 
books.” Master's thesis. Greeley, Colo.: Colorado State Teachers College, 1932. Pp. 75- 

181 Ray Albright, “The Development of Arithmetic Textbooks in the United States.” 
Master's thesis, Indianapolis: Butler University, 1933. Bpois3 

Carl E, Cole, “An Analysis of the War Content of Thirty Junior and Senior High- 
School American History Textbooks,” Master’s thesis, Minneapolis: University of Minne- 
sota, 1933. Pp, 96. 

Bessie Louise Pierce, Civic Attitudes in American School Textbooks. Chicago: Uni- 


132 Dorothy R. Mohr, “The Contribution of Ninth-Year Mathematics Textbooks to the 
Attainment of the Objectives of Ninth-Year Mathematics.” Master's thesis, Chicago: 
University of Chicago, 1933. Pp. 113. 

Frank Forchtner, “The Adaptations of Recent High-School Biology Textbooks to Cur- 
riculum Needs.” Master's thesis. Pierre: University of South Dakota, 1933. Pp. 48. 
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on the ro Criteria Set up by the Oregon State Textbook Commission.” Master's thesis. 
Eugene: University of Oregon, 1933. Pp. ror. 
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grade placement or general difficulty of the material.%* For 
numerous other examples, see the topic, “Textbooks,” in the 
annual Bibliography of Research Studies in Education pub- 
lished by the U. S. Office of Education. 

Some of these analyses are highly mechanical, but they have 
proved to be far from useless, In fact, studies of vocabulary 
burden have received considerable attention in recent years 
from textbook writers, and the other analyses have served to 
make authors more conscious of the quantitative aspects of 
their product. The appraisal of textbooks, which differs from 
textbook analysis, although it involves a certain amount of 
analysis, is discussed in the following chapter under the topic, 
“Appraisal Procedures.” ; 

Analysis of larger bodies of literature. In a study of “The 
Techniques of Research Employed in Arithmetic,” Brownell 1° 
canvassed 532 reports of investigations in this field, classified 
them with respect to problem and type of technique used, and 
then counted the frequency with which the different techniques 
have been used in the study of each kind of problem. Sixty- 
seven different kinds of problems and nineteen types of in- 
vestigational techniques were recognized. This study repre- 
sents well several distinctive characteristics of documentary 
frequency studies. It was based on an extensive body of litera- 
ture; it called for the making of categories appropriate to the 
study (e.g., the classes of problems and of techniques) ; and 
it presented the frequency with which each technique had been 
used for each study. Further, by a classification of problems 
and techniques in a two-way frequency table, the relationships 
between problem and technique were clearly portrayed. 


Third Semester Texts” Master’s thesis. Nashville, Tenn.: George Peabody College for 


Teachers, 1933. Pp. 112. 

Marion Esther Dalrymple, “A Study of the Present Use of Decimals in Industries, 
Periodicals, and Textbooks.” Master’s thesis, Boston: Boston University, 1933. Pp. 77- 

184 Jacob A. O. Lien, “The Grade Location and Drill Frequency of Certain Adjective 
Modifiers in Selected Language Texts.” Master's thesis. Iowa City: Iowa State Uni- 
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Brownell makes the interesting comment:1% “The blank 
spaces in these tables are fully as significant as are the spaces 
in which numbers have been entered.” They show that certain 
classes of problems have not been attacked by certain tech- 
niques and stimulate one to consider whether a contribution 
could be made in those areas. This same author made another 
frequency study of investigations in arithmetic and reading *°7 
showing the historical trend and the number of investigations 
per author. 

Documentary frequency studies have been employed to a 
considerable extent in basic curriculum work. In fact this field 
has received the most extensive application of the frequency 
technique. The underlying theory states that since not every- 
thing known can be taught in school, the things which are 
socially most useful should be selected and taught. These, 
according to the basic assumption, can be discovered by an ap- 
propriate interpretation of frequency analyses of social activi- 
ties and interests. 

An outstanding publication based on this point of view 
contains a number of studies made under the direction of 
Bobbitt.” One set of investigations was devoted te discovering 
the major fields of human concern and included an examina- 
tion of the frequency with which topics in different fields were 
listed in the Reader’s Guide to Periodical Literature for three 
years, a study of the column-inches of space devoted to various 
topics in a newspaper over a period of time, an analysis of the 
number of columns devoted to each topic in the Encyclopaedia 
Britannica, a classification of the topics apparently referred 
to by the ten thousand most frequently used English words, 
and a study of the topics treated in the Literary Digest. It will 
be observed that in each of these studies, the making of appro- 
priate categories was incumbent upon the worker. 


186 Ibid., p. 442. 

187 William A, Brownell, “Growth and Nature of Research Interest in Arithmetic and 
Reading,” Journal of Educational Research, XXVI (February, 1933), 429-41. 

188 Franklin Bobbitt and Others, Curriculum Investigations. Supplementary Educa- 
tional Monographs. No. 31. Chicago: University of Chicago, 1926. Pp. 204. 
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The monograph contains a number of other documentary 
frequency studies, including the following: An effort was made 
to determine the duties and traits of a good citizen by an- 
alyzing newspaper editorials and magazine articles on citizen- 
ship, in addition to opinions secured through interviews with 
leading citizens. Civic and social shortcomings were studied by 
an examination of the editorials of newspapers and magazines. 
The social problems of the labor group were determined by an 
analysis of ten widely used books and seven labor periodicals. 
A study of essays aimed to discover the characteristics of 
human behavior that are approved and disapproved; a canvass 
of books, magazine articles, and books of etiquette sought to 
determine approved social behavior. Shortcomings in the writ- 
ten English of adults were discovered by an analysis of letters 
written for newspaper publication. The science articles in a 
random sampling of five general magazines and three books on 
popular science were-studied to determine the mathematics 
used in popular science. The monograph suggests the wide 
range of opportunity for investigation in the field of general 
activity analysis as a guide to curriculum making.'*® 

Charters is another leader in the field of analytical fre- 
quency studies, often referring to his technique as “activity 
analysis” or “job analysis.” He discusses the principles of this 
technique in his early book, Curriculum Construction.*° Much 
of his work has, however, involved the collection of statements 
(“unrecorded specifics”) concerning details of what people do, 
so that basically it is more closely related to the questionnaire 
technique than to the present one. His work on the Common- 
wealth Teacher-fraining Study ** and on the analysis of 


189A publication representing Bobbitt’s earlier work in this field is: Curriculum- 
Making in Los Angeles. Supplementary Educational Monographs, No. 20. Chicago: Uni- 
versity of Chicago, 1922. Pp. 106. 

140 W, W. Charters, Curriculum Construction, New York: The Macmillan Co., 1923. 
Pp. 352. See Chapter IV, pp. 34-40, and the topic, “Job Analysis,” in the Index. 
, “The Use of Activity Analysis in Curriculum Construction,” Educational 
Research Bulletin, VIL (November 14, 1928), 339-42. 
, “A Technique for Trait Analysis,” Journal of Educational Research, X (Sep- 
tember, 1924), 95-100. 

141 Charters and Waples, op. cit.. (footnote 110). 
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secretarial duties and traits ™? has already been mentioned. 
One of his significant contributions in this field was the de- 
velopment of a pharmaceutical curriculum.** Aside from the 
fact that most of the data for his studies were recorded de novo 
for that purpose, the studies are similar to the others discussed 
in this section and represent the same general type of analysis. 

Analysis of assembled specimens. Not all of the material 
used in documentary frequency studies may be located in 
books and magazines. For certain studies one must collect his 
materials from sources other than the library. Such studies 
differ from the questionnaire investigations already discussed 
in that the materials have, in the present case, already been 
written for other purposes and are merely collected as speci- 
mens for the purpose of analysis. Three fields of study will be 
referred to as illustrations of this kind of research—vocabulary 
studies, error studies, and studies of record and report systems. 

Vocabulary analyses. For many purposes it is useful to 
know the most common words and the most readily under- 
stood words. In 1921 Thorndike took the lead in this field 
with the publication of his well-known Teacher’s Word Book,'** 
which ranks (by groups) the 10,000 most common words in 
order of their frequency. This list has recently been revised 
and extended.’*° Frequency lists of the most common words in 
several foreign languages have also been prepared.‘** Such 
lists 47 are important in establishing a common basic vocabu- 


142 Charters and Whitley, op. cit., (footnote 109). 

143W. W. Charters, A. B. Lemon, and Leon M. Monell, Basic Material jor a 
Pharmaceutical Curriculum, New York: McGraw-Hill Book Co., 1927. Pp. 366. 

144 E. L. Thorndike, Teacher's Word Book. New York: Columbia University, 1921. 
Pp. 134. (See also next footnote.) 

145 E. L. Thorndike, Teacher's Word Book, of the Twenty Thousand Words Found 
Most Frequently and Widely in General Reading for Children ənd Young People. 
Revised Edition. New York: Teachers College, Columbia University, 1932. Pp. 182. 

146 See for example, H. S. Eaton, “Comparative Frequency List Based on the First 
Thousand Words in English, French, German, and Spanish Frequency Lists,” in Experi- 
ments and Studies in Modern Language Teaching, compiled by Algernon Coleman, Chi- 
cago: University of Chicago Press, 1934. Pp. 367. 

147 For a bibliography, see “Modern Foreign Languages—Vocabulary Studies,” pp. 
156-8, in “The Curriculum,” Review of Educational Research, IV (April, 1934). An- 
other bibliography is contained in ‘Vocabulary Studies,” pp. 691-702, of The Modern 
Language Teacher's Handbook, by Thomas Edward Oliver. Boston: D, C. Heath and Co, 
1935. Pp. 706. 
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lary,“ both in reading and writing, and in the construction 
of readers and language textbooks.'*? The words which are 
regarded as technical, or peculiar to a particular school sub- 
ject, have been studied and compiled in mathematics,’ geog- 
raphy, history, *? and other subjects. Such lists make 
possible an estimate of the difficulty of these words and of the 
textbooks which contain them or go beyond them in difficulty. 
A criticism of their value has, however, been raised by 
Wesley.*** 

These studies have been based partly on printed literature 
and partly on collected specimens. A group of studies which 
have depended more heavily on collected materials are those 
connected with the spelling curriculum. Such studies are 
typically concerned with the frequency with which words are 
used in different situations, such as reading, writing letters, 
speaking, etc. They have been made in large numbers as the 
basis for spelling curriculums and spelling textbooks.’ 


148 “The Commonest Words in the Spoken Vocabulary of Children up to and Including 
Six Years of Age,” pp. 186-93, and also pp. 193-8, Report of the National Committee 
on Reading, Twenty-Fourth Yearbook of the National Society for the Study of Educa- 
tion, Part I. Bloomington, Hl.: Public School Publishing Co., 1925. Pp. 356. 

‘Arthur I. Gates, Reading Vocabulary for the Primary Grades. New York: Teachers 
College, Columbia University, 1926. Pp. 23. 

Clarence R., Stone, “The Second-Grade Reading Vocabulary,” Elementary School Jour- 
nal, XXXV (January, 1935), 359-67. 

149 For suggested uses, see E. L. Thorndike, “Word Knowledge in the Elementary 
School,” Teachers College Record, XXII (September, 1921), 334-70. See also more recent 
articles by Thorndike on reading and word knowledge, as given in the Education Index. 

For a critical analysis of Thorndike’s word list from the point of view of various uses, 
see Edgar Dale, “Evaluating Thorndike’s Word List,” Educational Research Bulletin, X 
(November 25, 1931), 451-7- 

150S. L. Pressey, L. C, Pressey, and R, C. Zook, “The Essential Technical Vocabulary 
of Plane Geometry,” School Science and Mathematics, XXXII (May, 1932), 487-9. 

Clifford Woody, “Nature and Amount of Arithmetic in Types of Reading Material for 
the Elementary Schools,” Educational Outlook, VI (May, 1932), 199-217. Also pub- 
lished as a Bulletin of the Bureau of Educational Reference and Research, No. 145. 
Ann Arbor, Mich.: Uniyersity of Michigan, 1932. Pp. 79. 

151 L, C. Pressey, “Fundamental Vocabulary in Elementary School Geography,” Jour- 
nal oj Geography, XXXII (February, 1933), 78-81. 

152 A. S. Barr and C. W. Gifford, “The Vocabulary of American History,” Journal oj 
Educational Research, XX (September, 1929), 103-21. 

158 Edgar B. Wesley, “Some Criticisms of Word Lists with Particular Reference to 
History,” Social Studies, XXV (February, 1934), 79-82. 

154 F, S. Breed, “Spelling,” Chapter X, pp. 83-8, in “Special Methods and Psy- 
chology of the Elementary-School Subjects,” Review of Educational Research, V (Feb- 
ruary, 1935), No. 1. 

F. S. Breed, “Selected References on Elementary-School Instruction: Spelling,” Ele- 
mentary School Journal, XXXIV (October, 1933), 137-9, and again, XXXV (October, 
1935), 139-40, and in subsequent annual issues in this series. 
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Horn *** has compiled the 10,000 words most commonly used 
in writing and includes in his report a critical evaluation of 
the various investigations underlying his work, as well as a 
critical review of previous similar investigations, The methods 
of building spelling vocabularies have also been reviewed 
critically by Coleman.’ Studies of the actual difficulty of 
words involve a different technique 1" and have not been so 
frequent. One who is interested in these fields should consult 
“Vocabularies,” and “Vocabulary Studies,” in the Education 
Index and also in the subject index of the annual Bibliography 
of Research Studies in Education of the U. S. Office of Educa- 
tion. 

By way of interesting contrast to these frequency studies, 
mention may be made of Ogden’s “Basic English.” 18 This 
system consists of a vocabulary of only 850 words which have 
been found to possess great power of expression. These words 
were not selected on the basis of frequency at all, but were 
chosen because each of them constituted an essential elemental 
idea out of which complex ideas could be built. Although the 
list is not intended for the same uses as the lists based on 
frequency, it is a striking example of some of the considera- 


See similar treatments in “Special Methods in the Elementary School,” Review of 
Educational Research, 1 (October, 1931), No. 4, pp. 276-9, and in “Psychology of the 
School Subjects,” Review of Educational Research, I (December, 1931), No. 5, pp. 341-2. 

C. T. Wise, “Selection and Gradation of Words in Spelling,” Elementary School 
Journal, XXXIV (June, 1934), 754-66. 

156 Ernest Horn, A Basic Writing Vocabulary: 10,000 Words Most Commonly Used in 
Writing. University of Iowa Monographs in Education, First Series, No. 4, April 1, 
1926. Iowa City: College of Education, University of Iowa. Pp. 225. 

156 William H. Coleman, A Critique of Spelling Vocabulary Investigation. Education 
Series, No. 12. Greeley, Colo.: Colorado State Teachers College, 1931. Pp. r19. 

157 V. H. Kelley, “Experimental Study of Certain Techniques for Test ing Word Mean- 
ings,” Journal of Educational Research, XXVII (December, 1933), 277-82. 

Guy T. Buswell and Lenore John, The Vocabulary of Arithmetic. Supplementary Edu- 
cational Monographs, No. 38. Chicago: University of Chicago, 1931. Pp, 146, 

Edgar Dale, ‘Difficulties in Vocabulary Research,” Educational Research Bulletin, X 
(March 4, 1931), 119-22. 

Edward W. Dolch, “Testing Word Difficulty,” Journal of Educational Research, 
XXVI (September, 1932), 22-27. 

158 A series of books have been published in connection with Basic English. The two 
following works are cited as leading examples; 

Charles K. Ogden, The System of Basic English, New York: Harcourt, Brace and Co., 
1934. Pp. 320, j 

The Basic Dictionary: Being the 7,500 Most Useful Words with Their Equivalents in 
Basic English, for the Use of Translators, Teachers, and Students. London: Kegan, Paul, 
1932. Pp. 106. 
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tions which are normally omitted from frequency studies. It 
also affords objective evidence that a word list of general im- 
portance can be built up entirely apart from a consideration 
of frequency. Those who are prone to rely largely on relative 
frequency should give thought to this list. 

The studies referred to in the preceding paragraphs have 
typically included the gathering of children’s compositions, 
social letters, and other examples of child or adult usage of 
words in everyday situations. Many of the studies have sup- 
plemented such information by counts of the words in pub- 
lished materials. Word lists have been referred to as illustrative 
of the technique of counting frequencies of occurrence in in- 
formal usage, and they are not meant to suggest that the 
technique is limited to this field. Study of the informal usage 
of arithmetic *** and other subjects has been undertaken, in 
addition to the analyses of the more formal usage occurring 
in printed sources, as typified by the curriculum studies of 
Bobbitt. 

Error studies. A different purpose has prompted another set 
of studies of informal usage. These investigations are con- 
cerned, not with the frequency of use, but with the frequency 
of errors made in usage. The method has been similar to that 
of the analyses of usage, except that the error studies have 
perforce depended more completely upon informal materials. 
These investigations have been devoted chiefly to language in 
its various phases, since errors in language (including reading 
and the mechanics of writing) normally constitute the bulk of 
the errors which are made in informal written material. 
Charters gave considerable impetus to this movement in its 
earlier days. Studies of errors have been made in considerable 
number, especially in the “usage” fields. Spelling, ® arith- 

159 For a pioneer study, see G. M, Wilson, “A Survey of Social and Business Uses of 
Arithmetic,” pp. 128-42, in the Second Report of the Committee on Minimal Essentials 
in Elementary-School Subjects. Sixteenth Yearbook of the National Society for the Study 
of Education, Part I. Bloomington, Ill.: Public School Publishing Co., 1917. Pp. 204. 


160 See the discussion and references in ‘Special Methods and Psychology of the 
Elementary-School Subjects,” Review of Educational Research, V (February, 1935), No. 


1, pp. 87-8. 
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metic,** geography,'®* modern foreign languages,’ capitaliza- 
tion, punctuation, grammar, sentence structure, speech,!** and 
other fields have recently been subjected to such analysis, in 
addition to many studies in earlier years. The errors made by 
teachers have also been studied in several cases.’ Fifty-eight 
of these error analyses are reviewed and classified as to tech- 
nique of gathering data by Brownell and Easley. 
Characteristics of reports. As a third field that illustrates 
documentary frequency studies, based largely on collected 
specimens, studies of school records and reports are referred 
to. Records and reports constitute a phase of administration, 
and also instruction, that is of basic importance, and many 
studies of them have been made. One of the predominant types 
of investigation has collected records or reports from a number 
of school systems and analyzed them. This analysis usually 
involves noting certain characteristics of the forms and then 
counting the number of school systems that have this char- 
acteristic (frequently an item on a blank) in their set of 
records or reports. An interesting table occurs when one counts 


161 Ibid., pp. 21-2. See also, “Psychology of the School Subjects,” Review of Educa- 
tional Research, I (December, 1931), No. 5, pp. 358-9. 

162 “The Curriculum,” Review of Educational Research, IV (April, 1934), No. 2, pp. 
155-6. See also footnote 165. 

163 Ibid., p. 159. 

164 Ibid., p. 202; also, “Psychology and Methods in the High School and College,” 
Review of Educational Research, IV (December, 1934), No. 5, PP. 449-51. 

Studies of errors in oral and written English up to 1929 have been reviewed in detail 
by R. L. Lyman in his Summary of Investigations Relating to Grammar, Language, and 
Composition, see Chapter III, pp. 71-133. Supplementary Educational Monographs, No. 
36. Chicago: The University of Chicago, January, 1929. Pp. 302. 

See also, Henry Harap, “The Most Common Grammatical Errors,” English Journal, 
XIX (June, 1930), 440-6. 

One study dealt with letters written for publication in a newspaper; these differ some- 
what from material usually studied in that they may be regarded as formal. See Sarah A. 
Bobbitt, “Shortcomings in the Written English of Adults,” Chapter XII, pp. 110-8, 
in Curriculum Investigations. (See footnote 138.) 

165 Edward U. Engleman and J. R. Shannon, “An Analysis of Teachers’ English 
Errors,” English Journal (College Edition), XXII (January, 1933), 45-52. 

Isabelle K. Hart, “A Classification of Common Errors in Geography Made by 
Teachers and Pupils,” in The Teaching of Geography, pp. 479-82. Thirty-Second Year- 
book of the National Society for the Study of Education. Bloomington, Il.: Public 
School Publishing Co., 1933. 

Walter S. Guiler, “Computational Errors Made by Teachers of Arithmetic,” Elemen- 
tary School Journal, XXXIII (September, 1932), 51-58. 

166 William A. Brownell and Howard Easley, “Analyses of Errors,” pp. 302-7, in 
“Psychology of Learning, General Methods of Teaching, and’ Supervision,” Review of 
Educational Research, II (October, 1933), No. 4. 
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the number of school systems that have certain numbers of 
records, resulting in a sort of double frequency distribution in 
which the scale numbers are themselves frequencies. 

These studies have not infrequently combined other criteria 
with their frequency analysis. For example, Heck +° secured 
four different frequencies and ratings in addition. In studying 
the pupil-record forms used in 131 cities, he ascertained the 
frequency with which each item appeared (i.e., the number of 
cities having this item on their forms), the frequency with 
which each item is needed by superintendents or others to pre- 
pare the required reports made to the state (reports from the 
forty-eight states were collected), the frequency with which 
each item would be needed to make various studies that have 
been listed in the literature as desirable, and finally, the fre- 
quency with which each item was actually used by teachers 
and principals. This information was then put in the hands of 
133 judges, who passed judgment on the value of each item 
for inclusion in a “universal list” of items for all school 
systems. 

Studies have been made of many different aspects of records 
and reports. Some workers have studied the superintendent’s 
report, as to its mechanical make-up," and as to its effective- 
ness in giving information and in properly publicizing the 
schools.’ Such investigations may follow the same pattern as 
textbook analysis. A number of studies have been concerned 
with the records of city school systems,” of county school 


167 Arch O. Heck. A Study of Child-Accounting Records. Bureau of Educational Re- 
search Monographs, No. 2. Columbus: Ohio State University, 1925. Pp. 245- Reviewed 
briefly in his Administration of Pupil Personnel, pp. 230-5. Boston: Ginn and Co., 1929. 
Pp. z 

Sies Emi Rinsch, “An Analysis of 56 City Superintendents’ Annual Reports.” Master's 
thesis. Bloomington, Ind.: Indiana University, 1927. Pp. 92. 

169 Robert Augustus Jacob, “The Superintendent's Annual Report Functioning as a 
Tool.” Master's thesis. Nashville, Tenn.: George Peabody College for Teachers, 1928. 


Pp. 64. 

rhe’ following study, though based on the frequency with which items appeared in the 
literature rather than in collected specimens, is mentioned because of its significance in 
this general field: Zenas Read Clark, Recognition of Merit in Superintendents’ Reports 
to the Public. Teachers College Contributions to Education, No. 471. New York: Teach- 
ers College, Columbia University, 1931. Pp. 123. 

170 Arch O, Heck, op. cit. 
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systems," and state school systems.‘ Other analyses have 
been devoted to particular forms or sets, such as the cumulative 
record card of pupils, the report card or other forms of report- 
ing to parents, and teacher’s class register, the records of the 
head of a department in high school, office records, the prin- 
cipal’s monthly and annual report to the superintendent, the 
superintendent’s monthly report to the school board, reports 
to colleges, financial reports, the annual report of the business 
manager, etc. Various studies have divided their consideration 
according to school level and reported on the records appro- 
priate for elementary schools, high schools, etc. 

References to these numerous studies will be found in the 
annual Bibliography of Research Studies in Education pub- 
lished by the U. S. Office of Education, beginning with 1926-27. 
The topic heads vary; one should look in the Subject Index 
(at the back) for “School Records and Reports,” “School Re- 
ports,” “Reports,” or “Records and Reports’; also, for “Child 
Accounting,” “School Attendance,” or “Attendance and Child 
Accounting.” These references are mainly to theses; studies 
reported in periodicals and elsewhere will be found in the 
Education Index under the topics, “Reports and Records,” 
“Report Cards,” and “Child Accounting.” Related topics are: 
“Census, School,” “Marking Systems,” and “Marks, Students’.” 
Look also for cross references to other topics. 

It is of course clear that frequency analyses may be made 
of other types of documents than those which have been here 
discussed. For example, Shank ** studied tests of silent read- 
ing to ascertain the frequency with which thirty-three different 
types of response were called for by the different tests, An- 
alyses of this type should prove fruitful. 

Related forms of study. It will be recalled that frequency 


171 Robert Oliver Nelson, “A Study of Southern County Superintendents’ Annual Re- 
ports.” Master’s thesis, Nashville, Tenn.: George Peabody College for Teachers, 1930. 
Pp. 75. 

172 Frank Leslie Shaw, State School Reports. Contributions to Education, No. 242. 
New York: Teachers College, Columbia University, 1927. Pp. 142. 

178 Spencer Shank, “Student Responses in the Measurement of Reading Comprehen- 
sion,” Journal of Educational Research, XXII (September, 1930), 119-29. 
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studies of other phases of administration are somewhat com- 
mon also, being made through the use of the questionnaire and 
check-lists. Ascertaining and analyzing current practices is one 
of the principal purposes of normative-survey research, and 
counting is a favorite procedure. 

There are of course other kinds of analysis that can be made 
of documents besides counting the frequency with which cer- 
tain aspects occur. For example, the field of historical work 
is dependent upon documents as sources, but makes an entirely 
different use of them. There are diaries or other forms of 
autobiographical records which can be analyzed for one pur- 
pose or another. Closely related to these are personal reports, 
written upon request or given orally, and usually involving a 
certain amount of introspection. Such material may be an- 
alyzed in a variety of ways, for different purposes. One instance 
which may be pertinent to the present general treatment is 
Blumer’s analysis of preliminary case reports,'"* for the pur- 
pose of finding the topics treated in them, so that he could use 
these topics in blanks which he was preparing for further re- 
ports, as suggestive outlines for the later reports. 

Interpreting frequency studies. The interpretation of fre- 
quency analyses based on existing materials is in many respects 
similar to the interpretation of questionnaire returns, and one 
should refer to the discussion of the latter topic. The same 
problems arise in obtaining practical significance from fre- 
quencies; that is, in determining the legitimate conclusions 
which can be drawn from frequency distributions. {n the case 
of curriculum work this problem has been clearly recognized 
and dealt with from several angles. Probably frequency is an 
element in importance, but it is only one element, and its 
significance must be evaluated by carefully analyzing its logical 
contribution to the total picture. We may note the comments 
made by Bobbitt in connection with the frequency analyses 


t Blumer, Movies and Conduct. New York: The Macmillan Co., 1933. Pp. 
o ana aeaii by W. W. Charters, Motion Pictures and Youth, p. 36. New York: 


The Macmillan Co., 1933. Pp. 66. 
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of literature which sought to identify the major fields of 
human concern: "7 


In the aggregate, man seems to prefer to dwell ən the little things 
that make up his existence from hour to hour and is reluctant to dwell 
on the things that are large and high and intrinsically important. He 
will attend to the large things when they reach the point where they 
worry him, but only because they worry him and not because of their 
inherent importance. .. . “Language” .. . is infrequently discussed. 
The fact that it is an absolutely indispensable factor of civilization 
does not place it high in the list....It is probable that “govern- 
ment,” which heads the list with a count of 9,920, is not more im- 
portant than is “language,” with a frequency of only 220. The political 
world is a seething ferment of problems, difficulties, worries, and fears, 
however, while language presents us with few problems, none of them 
threatening our very existence. 

Things bristling with immediate problems of which people are gen- 
erally aware are far more frequently discussed than are things which 
involve problems more remote and visible only to the specialized few. 
Problems of population, for example, are probably more serious when 
the long view is taken than are those of a political or economic char- 
acter. They are almost certainly our most threatening problems, but 
there is a general unawareness of them. Hence the frequency of men- 
tion is low, 

The foregoing discussion shows the impossibility of accepting the 
frequency figures as indices of the relative importance of the several 
fields. They seem to constitute one of many types of evidence which 
can be used in the evaluation of the fields. 


Such considerations make it clear that one cannot properly 
infer general importance directly from relative frequency. It 
does not follow, however, that frequency has no worth, One 
has to define carefully what he has in mind by “importance.” 
Importance for what? Facts vary in importance depending 
upon the purpose for which one desires to use them. One may 
properly infer from Bobbitt’s findings that a knowledge of 
problems and events in the realm of government is of large 
importance if one is interested in following with some under- 
standing and appreciation the literature that appears in cur- 
rent periodicals. To conclude that such a knowledge is im- 


175 Franklin Bobbitt and Others, Curriculum Investigations, pp. 9-11. Supplementary 
Educational Monographs, No, 31. Chicago: University of Chicago, June, 1926. Pp. 204 
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portant for any purpose, however, does not necessarily follow. 
To assert, for example, that such knowledge becomes the 
responsibility of the school to inculcate in each pupil, is a con- 
clusion that does not grow immediately out of the research 
findings, but depends on a chain of reasoning which involves 
many other considerations outside of the objective evidence. 
Whether relative frequency throws much or little light on one’s 
problem depends upon the relation of the factors which the 
data represent to the conclusions one desires to draw. 

Heck 17° recognized the dependence of importance upon pur- 
pose and secured data on four aspects of importance, In 
seeking to determine a universal, minimum list of items for 
school-record forms, he secured the frequency (universality) 
with which items appeared in the record systems of different 
city school systems. This frequency represents a composite of 
judgments of general importance on the part of many persons 
who prepared the forms in the different school systems. A 
second frequency value was determined for each item by an- 
alyzing state-report blanks, to see how many states called for 
each item. This is obviously important from the standpoint of 
meeting state requirements. A third frequency grew out of the 
need for items of information in order that important studies 
of school systems could be made. This frequency was deter- 
mined by analyzing lists of studies that were recommended in 
educational literature as being desirable. The fourth frequency 
grew out of direct use in the school system; how often were 
items needed by principals and teachers? It is obvious that 
each of the last three frequency lists throws light on the im- 
portance of an item for a particular purpose. The first fre- 
quency suggests judged importance. In order to relieve the 
reader of having to weigh all of these factors, Heck had a large 
group consider these four frequencies and interpret them as 
indicating a certain degree of “general” importance. Perhaps 
the four frequencies combined with the general judgment of 
the “jury” would yield a good estimate of general importance. 


176 Arch O, Heck, op. cit., (footnote 167). 
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Charters and Waples, in the extensive Commonwealth 
Teacher-Training Study,’ secured only one series of fre- 
quencies, but they secured ratings on three other aspects of 
each teacher activity in their list; namely, importance (de- 
sirability or cruciality) for good teaching, difficulty of learning 
to perform the activity, and appropriateness of learning the 
activity in a training institution. Note that importance was 
directly rated, as a product of group judgment and was not 
deduced from frequency. The relative importance of providing 
for teaching an activity in a teacher-training institution would 
be decided after a consideration of all four sets of evidence 
on the activity. Such additional data lessen the strain upon 
frequency when it is the sole basis for conclusions. 

Attention should be given to the concept of “frequency.” It 
is commonly used in two different senses. If one secures re- 
sponses which indicate the number of people who do a certain 
thing or the number of times it occurs throughout the group, 
the frequency may be interpreted in the sense of proportion 
of the group, or as universality. It is in the nature of a per cent. 
On the other hand, frequency may refer to a time concept; 
ie., how often does a single individual do something? This 
figure should be thought of with reference to an average in- 
dividual in the group. In the Commonwealth Teacher-Training 
Study both types of frequency were secured; e.g., each teacher 
reported the activity as performed very frequently, infre- 
quently, or with intermediate frequency (as defined in the 
check-list). Tyler 1° studied the relationships between univer- 
sality and temporal frequency and found them to be somewhat 
related. Of course an activity which an individual never per- 
forms counts as zero both in the sense of the per cent of the 
group reporting that activity and in the sense of the frequency 
with which an individual does it. The more items, therefore, 
which are in the list which individuals do not do, the more the 

177 W. W. Charters and Douglas Waples, The Commonwealth Teacher-Training Study. 
Chicago: University of Chicago Press, 1929, Pp. 666. 


178 Ralph W. Tyler, “The Relation Between the Frequency anà the Universality of 
Teaching Activities,” Journal of Educational Research, XXII (September, 1930), 130-2. 


| 


THE NORMATIVE-SURVEY METHOD 359 


two types of frequency tend to merge, and the closer the rela- 
tionship will be between them. 

A second large question concerning frequency studies as 
related to curriculum work is raised about their influence on 
progress. The criticism is that they necessarily find out the 
commonplace or the average, and this does not provide a suit- 
able goal. On this point Bobbitt comments: *° 


Education has no function except that of leading persons to perform 
properly the activities which constitute an enlightened, humanistic 
civilization, Our first task is to find out with definiteness and certainty 
what these activities are. The problem of determining the activities is 
not to find out what is usually done. It is not to find the average per- 
formance, The frequency of an activity on the part of a mediocre 
generation is not of much value in showing us at what education ought 
to aim. Those who have achieved the highest and most desirable levels 
of human performance are relatively few. The activities of the high 
type to be aimed at are therefore relatively infrequent. Let us ‘ind, 
therefore, if possible, those persons who come nearest to living life 
as it ought to be lived. 


From these remarks it is evident that Bobbitt would select 
his cases at the beginning, and that the element of evaluation 
enters into his research plans at the beginning of the work 
more prominently than at the close. è 

Charters and Waples make the following statement.: **° 


The criticism has commonly been raised against activity analysis that 
it determines duties that are performed rather than duties which 
should be performed. While this criticism is widespread, it is not well 
taken. 


These writers point out that in the Commonwealth Teacher- 
Training Study the master list of teacher activities was re- 
ferred to a number of experts with the request that they add 
to the list all of the duties that they thought ought to be per- 
formed by teachers. Progressive literature was also examined 

179 Franklin Bobbitt, op. cit., (footnote 175), p. 2- 

180 W, W. Charters and Douglas Waples, op. cit., pp. 120-5. 
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to the same end, In the evaluation of the activities, provision 
was made for determining frequencies of performance and 
ratings on importance, etc., by teachers in progressive experi- 
mental schools and by professors of education. These groups 
should represent forward-looking tendencies. The returns from 
these two groups were reported separately ** so that one may 
be guided by their data or by the more typical average data, as 
he prefers. 

A third large question relating to frequency analyses as 
bases for curriculum work centers around the aspects of learn- 
ing which they tend to emphasize. That is, such studies tend 
to be concerned with details; most educators say, at least, that 
they desire to place their emphasis upon integrating principles 
and general orienting ideas. On this point we may quote again 
from Charters and Waples. In commenting upon the use of 
their evaluated list of teacher activities as an important refer- 
ence in the construction of curriculums for teacher-training 
institutions, they say : *S* 


Without in any sense wishing to argue that a course for prospective 
teachers should be confined to the study of the significant activities in 
teaching, one may note in the foregoing list many activities that most 
inexperienced teachers need to have analyzed and explained at some 
length. If the index of curricular value is as valid as we believe, it 
would follow that at some point in the training program the significant 
activities should be described, together with the problems to be avoided 
or solved and the theoretical principles needed to give coherence and 
unity to the course. It may be seriously questioned whether any treat- 
ment of the principles that govern the performance of these activities 
can be highly effective unless the application of the principles to the 
activities is made explicit. 


B. R. Buckingham has called attention to some of the short- 
comings of frequency studies, in an address which he has given 
repeatedly to educational groups. Without contending that the 
study of frequency of use is unimportant, Buckingham points 


1s1W. W. Charters and Douglas Waples, op. cit, (footnote 177), Table C, pp. 
564-92, groups L and M. 
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out that it has serious limitations and should not, therefore, be 
made the sole or the chief basis of curriculum construction. 
The results of frequency counts bring highly heterogeneous 
concepts together in juxtaposition with the result that the find- 
ings lack any system or organization, an essential element in 
well-rounded mental development. Obviously the material is 
not related to the individual interests of the learner. The pro- 
cedure is likely to restrict usage still further by limiting in- 
struction to the commonplace, instead of attempting to bring 
into the experiences of ever larger numbers those educational 
contributions that will enrich their lives and broaden their 
mental powers. In Buckingham’s words, “Education contem- 
plates something more than the round of common experience, 
something richer and more joyous than the repetitious ele- 
mental facts of life.” 

His criticisms agree with other lines of evidence in emphasiz- 
ing that frequency studies need a large amount of supple- 
mentary interpretation if they are to be put to practical use 
with profit. Perhaps the full contribution which frequency 
studies can make must await the more complete development 
of a procedure for properly utilizing the findings in a frame 
of reference which recognizes other criteria of importance be- 
sides frequency, which includes the psychology of learning, and 
which gives due consideration to integrating principles, balance, 
and orientation. One of Buckingham’s points of criticism is 
that the technique has not yet developed a philosophy which 
will properly interpret the findings. 

Something should be said about the interpretation of error 
studies, such as those made in English usage and in arithmetic. 
These analyses may appear at first to have more intrinsic value 
than other frequency studies, for they deal directly with de- 
ficiencies. They are made for the purpose of revealing weak- 
nesses; one may feel that their significance is plain and 
incontrovertible. They suffer, however, from much the same 
types of limitation as do other frequency studies. 

In the first place such investigations do not reveal why the 
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child made the errors which are noted, and hence they afford 
little that is helpful! in teaching the child. An error is not to be 
looked upon simply as an error; it is a response habit that 
probably has a long history, growing out of a complex back- 
ground of information and associations which at some place 
or other have become tangled. To correct the error economically 
and permanently requires locating the point where the trouble 
is and giving the child a new insight before proceeding to drill 
him. In this connection Brownell and Easley comment: *8° 


Too many of these studies represent ends in themselves and make 
no contribution to an understanding of the learning process or to the 
direction of the teaching process. .. . Too frequently the data con- 
tain no suggestions as to why children encounter these difficulties, as to 
the precise nature of the difficulties, and as to how these difficulties 
may be avoided. 


In the second place, there is an implication in these error 
studies that whatever children do not know they should know. 
Tt may take several minutes of thought for the full significance 
of such naiveness to be realized. There is little value in know- 
ing what a child cannot do unless we also know what he should 
be able to do. We are interested in knowing what a child’s 
habits should be for each age and for each grade. Some error 
studies have given attention to this phase; the majority have 
not. There is no inherent value in comparing the habits of 
children—or, for that matter, the informal habits of adults— 
with the academically perfect formalities that the more in- 
tellectual members of the race have found delight in codifying. 

In the third place, typical error studies give no suggestion 
as to the psychology of learning—no hint as to how the ma- 
terial that should be learned may be worked into the child’s 
experiences so that it will contribute to, and become a part of, 
those larger patterns of response which give to language (or to 
reasoning processes) significance and effectiveness. In reviewing 

183 William A. Brownell and Howard Easley, “Psychology of Learning, General 
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studies of errors in written English, John Paul Leonard is 
prompted to comment: *** 


It is very doubtful if we can ever arrive at a satisfactory teaching 
program by basing our selection of items upon studies of the errors 
which children make. . . . Studies of the functioning of the child’s 
mind as he creates speech and writing are far more valuable than con- 
tinued error counts. i 


In the fourth place, by their very objectivity, the findings of 
such studies tend to distort the true picture of the child’s needs. 
They direct attention to certain very narrow aspects of his 
under-development and fail to reveal the condition of many 
other kinds of deficiency which probably exist in the pupil’s 
abilities or habits, and which may be of far greate: import. 
After all, we desire good habits not only in details but in one’s 
larger adaptations, and the latter are far more difficult to build 
up. It is, furthermore, difficult for the teacher to keep her at- 
tention on them, since they are less tangible, less easily held 
before one’s attention for effective consideration. From the 
practical point of view, there is positive danger in the emphasis 
that naturally grows out of error studies of the typical sort. 
A perspective that begins with errors rather than with a more 
complete picture of the accomplishment which is desired seldom 
reaches the more subtle, less conspicuous, but tremendously 
important aspects of ability, such as interest, pleasure in doing 
or using, emotional adjustment, purpose, and other large in- 
tegrating and dynamic patterns of performance. 

The foregoing discussion of interpretation of frequency 
studies has been detailed because the authors believe it is im- 
portant for students to realize that there is more to research 
than simply gathering data. The data must be selected and 
gathered for a purpose, and they must contain those factors 
which, when analyzed, will throw light on that purpose. There 
is little use in gathering data which cannot be interpreted, or 
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which, when legitimately interpreted, have no bearing upon 
one’s problem. The selection and gathering of data which are 
significant must be done in terms of the desired interpretation. 
Some of the difficulties of interpreting frequency counts have 
been pointed out. The writers are inclined to suggest that the 
simpler and the more objective the facts, the more circumspect 
one has to be in drawirig practical conclusions from them. 

The forming of categories. The chief technical problem of 
frequency studies is the formation of appropriate categories in 
which frequencies may be secured. The source of difficulty in 
these studies does not lie in the recognition and counting of 
separate objects; usually these are sufficiently discrete units 
for easy identification, such as pages, lines, errors, etc. The 
difficulty lies in determining suitable divisions among the larger 
or more general characteristics of these units; that is, in decid- 
ing upon, characterizing, and delimiting the classes into which. 
these units shall be placed. The problem is then, what kinds of 
pages, lines, errors, etc., shall be counted, and what groups shall 
be kept separate among them? 

The formation of appropriate categories is a rather general 
characteristic of research. It is essentially a process of noting 
similarities and differences, delimiting concepts, and fitting 
them into an integrating scheme. If one analyzes research in 
general, he will see how these processes underlie most of the 
steps. Choosing a problem involves delimiting the field of work ; 
selecting factors for study and sources of data, defining units 
for measuring or enumerating data, and choosing aspects of 
the collected data as a basis for analysis, all involve the proc- 
esses of perceptual and conceptual differentiation and synthesis. 
In wrestling with the problems of securing satisfactory cate- 
gories for data-gathering instruments, therefore, one is, in 
effect, epitomizing the whole of research. 

In a certain sense, all of the normative-survey techniques 
may be said to depend directly upon the use of suitable cate- 
gories. The items in a test are selected aspects of a more general 
set of abilities, which they are to sample. Questionnaires and 
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check-lists consist of aspects of some particular set of condi- 
tions, these aspects being set up as categories within which 
returns will be summarized. The methods to be discussed in the 
next chapter—the interview, direct observation, and appraisal 
—also have distinct categorical elements. The preparation of 
categories is, however, a particularly prominent part of fre- 
quency studies, and is therefore given special attention at this 
point. 

The worker encounters the problem of categorization from 
two different approaches, The first of these, and the simpler, is 
the choosing of those aspects of any group of objects which one 
desires to study to see how these aspects vary from one object 
to another. For example, if one is to compare different text- 
books, what aspects of textbooks will he select to study? There 
are of course many objective aspects of textbooks, such as 
chapters, paragraphs, lines, words, pictures, and the like. He 
can count these, but when he considers doing so, he should ask 
himself, “By what manner will I have advanced the under- 
standing and competence of educators when I have counted 
these things?” Or, somewhat more narrowly, “Will the count- 
ing of these things answer the basic questions involved in the 
problems of my research?” 

If one considers the less objective qualities of textbooks, such 
as. style, scholarship and authenticity, scope and breadth of 
treatment, emphasis, interest, articulation with the experiential 
background of a particular group of pupils, and the like, he 
begins to realize the many characteristics which a textbook has. 
How shall he choose those aspects which are significant for his 
purpose? Are there other aspects that would be still more sig- 
nificant if he could think of them? And (practically), how can 
he measure or count them after he has thought of them? 

The goal of this first type of category forming is to choose 
those aspects (categories) of the objects being studied which 
will represent the factors necessary to afford a satisfactory 
answer to the principal questions which are raised in the prob- 
lem. Or, looking at it in the other direction, the purpose is to 
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analyze the problem into components which are observable in 
the group of objects to be studied. One wishes to secure evi- 
dence on a set of aspects which, when taken together, will pro- 
vide a true cepresentation both of the objects and of the general 
variable constituting his problem. To secure these one must 
study both his problem and the objects. We speak of “choos- 
ing” or “selecting” aspects; in reality the process is more than 
this, for one must perceive in his problem and in the objects 
those elements which form quantifiable aspects, and, at the 
same time, embrace enough of the essential characterizing ele- 
ments of each so that elements of crucial importance will not 
be omitted. 

Much of the difficulty with mechanical research lies at this 
point—either in accepting a problem which permits of nothing 
more than routine study, or of seeing nothing more in the prob- 
Jem than what a “wooden” interpretation of it affords, ‘The 
research consumer who is charged with the responsibility for 
managing the practical affairs of education may not be able to 
put his finger on what is lacking in certain research studies, 
but he is in many cases certain that the conclusions are not 
sound, Often the trouble is that the investigator failed to see 
in his problem and in his cases some of the essential elements, 
without which his conclusions become relatively insignificant, 
if not misleading. If one is ingenious enough to formulate the 
appropriate categories, even counting words might tell him 
which textbook was the most engaging, the best written, the 
most suitable, etc.; the problem is to break down such general 
concepts into classes of words and word groups so that, when 
the words within these different classes are counted, the totals 
will have telling significance. 

This first type of categorization is concerned chiefly with an 
analysis of the problem, but in terms of characteristics of the 
field. It yields rubrics under which data are gathered. It is the 
essential process in constructing tests, questionnaires, check- 
lists, and other schedules which are or might be prepared in 
advance of receiving the data. The second type of categorization 
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is somewhat the reverse of the first. It is concerned chiefly with 
an analysis of the field, but in terms of, or from the point of 
view of, the research problem. It also yields captions within 
which data may be placed, but it does so only after specimens 
have been collected from the field, analyzed, and classes made 
which will fit them and arrange them into logical relationships. 
It may be said that the first type is essentially a division of 
the problem (or the general variable constituting the problem) 
which will fit the field, and the second type is a division of the 
field which will fit the problem. It is more important that the 
first be complete with respect to the problem, and that the 
second be complete with respect to the field. The second type 
is probably the more difficult and, fortunately, seems to be 
required less often. 

An example of the second approach to category formation is 
furnished by the analysis of literature (referred to earlier in 
connection with the curriculum studies sponsored by Bobbitt) 
for the purpose of ascertaining the major fields of human con- 
cern. These major fields were not set up in advance; they 
emerged from an examination of the literature. It is true that 
they were used to gather the frequency of occurrence of each 
one, but they grew nevertheless out of the analysis of the field 
and could not have been properly set up in advance. Similarly, 
a codification of errors found in informal manuscripts will be 
based principally upon the types of errors found, not upon an 
a priori assumption—unless one has built up his categories 
from previous experience with the same type of material. These 
statements do not mean that one cannot start out with some 
tentative categories, but they mean that the categories are 
properly indigenous to the field being studied. 

Categorization of this second type is at best complex, and 
there are no simple rules to make it easy. It may be helpful to 
refer to certain criteria. Three of these will be mentioned, 
namely, purpose, differentiation, and homogeneity. The pur- 
pose of the research study must be satisfied by the categories 
which are established. This purpose should be the determining 
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factor in choosing between many different possibilities of divi- 
sion. Differentiation relates to the distinctions between the 
categories; they should be different; they should not overlap; 
and the basis of differentiation should be clear and reasonably 
consistent. Homogeneity relates to the opposite characteristics 
of the classes. All of the cases which fall within any single 
category should be satisfactorily similar; that is, they should 
possess enough characteristics in common so that they belong 
together. In addition, the entire set of categories should possess 
enough common elements so that they can logically be re- 
garded as belonging in the same series. 

To produce a classification scheme which satisfies these 
criteria usually calls for a great deal of trying and readjusting. 
Although selection of bases for classification is to be made in 
the light of one’s purpose, it is not always easy to tell how dif- 
ferent bases will suit one’s purpose, and to begin with, one has 
to perceive a variety of bases for classification before he can 
select from among them. With the multiplicity of aspects which 
ordinary phenomena present, it is true that classification could 
proceed on almost any basis. For example, the animal kingdom 
could be classified, not on the basis of structure, but on the 
basis of weight, or length of life, or numerical predominance, or 
other bases. One will probably say that such bases would not 
be satisfactory for science, and perhaps they would not. But 
before one can judge the relative satisfactoriness of any basis 
for his division, he must see a number of different possibilities. 
Then he has to try out several of these possibilities to see 
whether the characteristics on which he would like to make 
divisions can be observed satisfactorily, whether they furnish 
divisions that are useful for his purpose, etc. 

A fair way to begin classification is to approach it empirically 
instead of theoretically. One may begin his classification by 
noting the cases (objects or phenomena) which seem to be 
different and to belong appropriately to different groups. One 
may, in fact, make up a classification scheme by merely writing 
down the names of objects of his study which experience or 
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cursory examination has indicated to him are different. When 
he has proceeded on this basis for a while, he will soon run into 
difficulties of classification which will force upon him a critical 
analysis of his attempts. Out of this examination should come 
a clearer idea of the basis he is using to differentiate his classes, 
as well as a refinement in the classes which he has temporarily 
set up. 

Perhaps as pure theory the forming of categories would not 
be commenced until all of the specimens had been collected. As 
a practice, however, this is frequently impossible and is prob- 
ably of little consequence. It is reasonable to gather a few data 
(specimens) or to draw upon one’s experiences if they are suf- 
ficient, and to attempt a preliminary classification on this basis. 
One may then gather more data and try out his tentative 
scheme on these. If it works, well and good; he will gather 
more data. Usually, however, he finds that, in the light of new 
data, his early scheme must be modified. He may need only to 
add new categories; he may find that he must reconsider his 
basis of classification; and he is likely to find that he needs 
to re-define some of his classes and change their limits. The 
process of categorization thus continues, with the gathering of 
a few data, the formulation of a scheme, and then the adjust- 
ment of the scheme to accord with the facts observed. It is 
understood that in all of these steps, one will constantly recall 
the demands of his purpose and that tentative outlines and 
subsequent adjustments will be made in the light of such needs 
as well as in terms of the data. 

The steps taken in the Commonwealth Teacher-Training 
Study provide an interesting illustration of building categories 
to classify a large number of items. In the study of teachers’ 
traits, an initial list of 2800 trait-actions was assembled. These 
were then grouped (“translated”) under eighty-three different 
traits which were either mentioned or selected to represent all 
of the trait actions. Finally, these eighty-three heads were re- 
duced to twenty-five heads by combining (“telescoping”) those 
which seemed to be reasonably similar. (The number twenty- 
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five was selected arbitrarily to provide a sufficiently detailed 
list for practical purposes.) Both the “translating” and the 
“telescoping” were done on the basis of majority judgment of 
a group of persons. 

In the study of teachers’ activities, a sample of some 12,000 
activities was drawn from the collection of 2 35,000 statements 
and was used as a basis for building up a classification. This 
proved to be a formidable problem. Charters and Waples 
report ; 18° 


Experiments to determine an efficient method of classification ex- 
tended over a period of approximately eighteen months and occupied 
the major attention of the entire staff. In general, it may be said that 
three types of classification were attempted. The first was a structural 
classification, consisting primarily in the alphabetical classification of 
nouns reported on the blanks. The second was a functional classifica- 
tion, represented by selecting the verbs which were reported on the 
blanks. The third method of classification, and the one finally adopted, 
combined the other two methods. That is to say, while the main head- 
ings of the classification represented the essential functions of teach- 
ing and consisted mainly of verbs, the subdivisions represented the 
structural elements of teaching as supplied by the nouns. Vet it is not 
entirely accurate to say that the classification finally adopted is a 
purely mechanical combination of nouns and verbs, It is more ac- 
curate to say that the total meaning of the activity statement had 
been reduced to its essential terms, 


The procedure adopted was then not a mechanical following 
of any plan. The same final scheme might have been arrived 
at by a general analysis formally considering and examining 
a basis of differentiation, for no single basis was held to. The 
concepts referred to in the quoted paragraph above underwent 
normal modifications that were required to make them more 
workable. The large divisions became differentiated on the 
basis of the individual with whom the teacher came in contact, 
such as pupils, other professional workers, and the community. 
But obviously a teacher has relationships with the physical 
environment, and relationships with abstract concepts (for 
example, organizing subject-matter), and so divisions had to 


185 W, W. Charters and Douglas Waples, op, cit., (footnote 177), pp. 81-2, 


THE NORMATIVE-SURVEY METHOD 371 


be included for these, Again, a teacher’s activities in connec- 
tion with her professional advancement are fairly distinct, but 
these are not thrown into a separate group by any of the fore- 
going bases of classification. To adapt the classification scheme 
to practical interests, a division was included for this group 
of activities, thus introducing the element of purpose into the 
classification. Furthermore, it was found desirable to break up 
relations with pupils into three main divisions and to combine 
other sets of activities with them in two of the main divisions. 

The resulting divisions therefore represent a mixture of 
bases of differentiation and homogeneity, in order that they 
might conform to groups that seem natural in our everyday 
thinking. Attention is called to these facts, not by way of 
criticism of the classification in this study, but rather to point 
out that what happened in this process is typical, and one 
should not only expect it but should adapt his working plans 
to that expectancy. If one attempts to select a single basis of 
differentiation and actually. holds to it rigorously, there is grave 
danger that his resulting categories will prove satisfactory to 
no one, for the concepts of “groups” and of “different groups” 
which are built up in our everyday thinking do not usually 
depend upon variation in any single factor. 

Probably the chief difficulty of classification grows out of 
the fact that objects do not belong to any single system of 
classes, but to an intricate network of systems. When we speak 
of bases of differentiation, it must be recognized that what we 
really use in a classification scheme that is at all elaborate is 
a hierarchy of bases. It may be suggestive to say that we 
normally differentiate in more than one plane, and that we 
might differentiate in an indefinitely large number of planes. 
By way of illustration we may refer again to the classification 
of the animal kingdom. Although the general basis of classifi- 
cation may be denoted as that of structure, such a generality 
does not disclose the division between the different phyla, or 
classes, orders, families, genera, and species, on the same hier- 
archical level, or between the different levels. In building up 
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categories, one has to decide when a difference is sufficiently 
great so that a new group on the same level is called for, or 
so that a new level is called for. If one could see in objects 
only the characteristics which differentiate them with respect 
to any one level, and this level were appropriate to his purpose, 
his problem of classification would be simple. But what one 
really sees is a host of characteristics belonging to different 
levels of a hierarchy—perhaps of several different hierarchies 
at once. He sees objects differing in several respects and similar 
in several others, and he does not know immediately whether 
these differences should be ignored or should constitute a 
separate class or a new level. Therefore, one has to classify the 
various characteristics of a set of objects into different levels 
of generality, to select those that belong on a level (or on 
levels) suitable for his purpose, and to use these, disregarding 
other similarities and differences which are judged not to be of 
immediate significance. 

One does not follow such a process rigorously, because it is 
much too great a task for the ordinary study. Ordinarily one 
attempts to accomplish approximately the same result by 
short-cut methods. Although this is normal, it calls for a critical 
examination of the work at every step to see that it is de- 
fensible. In examining his tentative products, one should con- 
sider ordinary ideas of natural groups, as well as abstractly 
logical differences. Many practical classification schemes in 
everyday use are not highly logical, but seem to work with a 
high degree of satisfactoriness. For most purposes we want 
something that is serviceable rather than theoretically finished, 
and the student should strive in the direction of the practical 
—or he will probably never complete his study. As examples 
of practical classification one may study the tables of contents 
of encyclopedias, textbooks, or other thought-skeletons of 
organized material. 

In closing this topic, it should be said that classification is 
a mental phenomenon and not an objective one. Objects differ, 
but the grouping of these objects into classes is a product of 
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the mind. Recognition of this fact will show that classification 
is associated with purpose as it is with objective facts. The 
facts alone do not indicate the scheme of classification; this 
must grow out of the mind of the worker, considering the facts 
in the light of his purpose. As a final caution, it should be re- 
peated that what one finds as a result of his study may be 
determined largely by the particular categories which he sets 
up; if these are satisfactory, his results may be representative ; 
if the categories are poorly set up, his results may be entirely 
misleading. 

Barr recognized these difficulties of classification in connec- 
tion with the determination of a working set of objectives for 
supervisors and proposed a method which utilizes the judg- 
ment of several workers in classifying objectives that have 
been proposed by others.'** 

Concluding statement. It was pointed out in the first part 
of this chapter that normative-survey research is designed to 
secure information on conditions or practices as they exist. 
The type of information which this kind of research yields is 
in wide demand, particularly by persons who have practical 
problems to solve. The method forms a useful supplement to 
other methods of research in affording knowledge of condi- 
tions over a wide area, and the procedures used in gathering 
data for normative-survey purposes are also utilized by other 
research methods. 

Three types of normative-survey research have been treated 
in the present chapter. Survey testing is used to ascertain the 
general level of a group with respect to school achievement, 
intelligence, or personality. Survey testing is carried on in 
large volume both within city school systems and in larger 
areas by state or regional agencies. Questionnaire inquiries are 
also frequent and may extend over wide areas. They are 
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adapted both to gathering facts and to gathering opinions and 
attitudes and have a wide range of application. Documentary 
frequency studies represent a quantitative analysis of written 
or printed material by counting the frequency with which 
specified characteristics occur. The method has received ex- 
tended use as a basic step in curriculum construction, through 
analyzing uses or goals, and errors or shortcomings. The 
method affords a survey of what is going on, or what has taken 
place, in the field of written and printed materials, much as 
the questionnaire does for information and attitudes which 
may not be expressed in writing. 

Normative-survey data are not generally easy to interpret. 
They require careful evaluation and thoughtful analysis in 
order that their true significance for practical conclusions may 
be discovered. 

A discussion of the construction of appropriate schedules for 
collecting data has been given in connection with each of the 
three types discussed, and references have been made to the 
literature which will be helpful in carrying out the technical 
details of preparing to undertake research. Typical studies of 
each kind of research have been cited so that those who are 
interested may extend their familiarity with the methods to 
any point desired. 

Three additional types of normative-survey research will be 
similarly treated in the chapter which follows. 


PROBLEMS AND EXERCISES 


1. Prepare a brief description of two research studies illustrating 
each of the three types of normative-survey research described in this 
chapter. Describe the similarities and differences existing between the 
studies representing the different techniques. These studies may be 
taken from the footnote references. 

2. Prepare a questionnaire on some subject of your selection, and 
present it to the class for detailed criticism. Preferably it should be 
mimeographed and filled in by each member, who should raise all of 
the questions possible about the form. The questionnaire might be 
worked out as a committee or a class project, and submitted to other 
persons for criticism. There should be class discussion of the results. 
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3. As an exercise in making categories, divide up the functions of a 
city school system, and arrange them in some systematic scheme of 
classification. 

4. To illustrate the difficulties met in a simple problem of classi- 
fication, group the forty-eight states into classes, on any appropriate 
basis, for any specified purpose, and defend this classification before 
the class. States may or need not be adjacent within any one class, as 
the instructor directs. In classifying and discussing, consider the 
homogeneity of each class, the differences between the classes, and the 
variations existing within a single state. 

5. Assume that you wished to make a study of how high-school 
pupils spend their entire time, and that you had secured the codpera- 
tion of roo pupils who were willing to report to you for a month. De- 
vise a set of categories in which you would like for them to report. 
Choose your own purpose for making the study, and then discuss the 
categories in the light of the purpose. Better still, in advance of the 
discussion, exchange classification schemes with other members of the 
class, and attempt to record your own time for two days in the cate- 
gories which have been devised. 
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CHAPTER VIII 


THE NORMATIVE-SURVEY METHOD: INTERVIEW, 
OBSERVATION, AND APPRAISAL PROCEDURES 


Introduction. This chapter continues and concludes the dis- 
cussion of normative-survey research. The preceding chapter 
dealt with three types: (1) survey-testing studies, (2) question- 
naire inquiries, and (3) documentary frequency studies. The 
present chapter deals with the second series of three types: 
(4) interview studies, (5) studies of behavior employing ob- 
servational procedures, and (6) survey-appraisal procedures. 
The last type, because of the quantity of material and variety 
of techniques, is treated in two divisions: (a) appraisal pro- 
cedures which involve direct judgment, and (b) appraisal 
procedures which are relatively objective. 

The three types of normative-survey research treated in the 
preceding chapter lean rather heavily upon prepared instru- 
ments, or schedules—tests, questionnaires, and categorical 
lists. The three types discussed in the present chapter— 
excepting 6-b—are more dependent upon the direct judgment 
of the worker at the time of gathering the data. The techniques 
in the 6-b group, like those of the preceding chapter, depend 
upon judgment primarily in the decisions which are made 
nrevious to the time of gathering the data. 


4. STUDIES EMPLOYING INTERVIEW PROCEDURES 


ín the interview for investigational purposes the research 

worker is gathering data directly from others in face-to-face 

contacts, as contrasted to most of the procedures heretofore 

discussed. Several kinds of facts can be secured only from 
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other people—for example, many facts of personal history, 
of certain personal habits and characteristics,? of family life,’ 
opinions, beliefs, etc. Some of these facts can of course be 
ascertained through the instrumentality of a questionnaire, 
check-list, or test distributed to groups of people. For that 
reason some workers refer to the interview as an “oral ques- 
tionnaire.” Such a reference or definition, however, may tend 
to obscure the unique characteristics of the interview, for it 
is indeed more than an oral questionnaire. 

By means of the interview it is possible to secure many 
data that cannot be obtained through the less personal pro- 
cedures of distributing a reply blank. People generally do 
not care to put confidential information in writing; they may 
want to see who is getting the information and receive guar- 
antees as to how it will be used; they need the stimulation 
of personal contacts in order to be “drawn out,” and some 
may be too ignorant to read and write. Furthermore, the 
interview enables the worker to follow up leads and take 
advantage of small clues; in complex material where the 
development is likely to proceed in any direction, no pre- 
pared instrument can perform the task. Again, the interview 
permits the interviewer to gain an impression of the person 
who is giving the facts, to form some judgment of the truth 
in the facts, and to “read between the lines,” things that 
are not said.* Finally the interview affords an opportunity 
for the interviewer to give information and to develop cer- 
tain attitudes on the part of the respondent. Such an oppor- 

1Ruth S. Cavan, “Interviewing for Life-History Material,” American Journal of 
Sociology, XXXV (July, 1929), 100-15. 

E. T. Krueger, “The Technique of Securing Life-History Documents,” Journal of 
Applied Sociology, IX (March, 1925), 290-8. 

“Arthur T. Jersild, Frances V. Markey, and Catherine L. Jersild, Children's Fears, 
Dreams, Wishes, Day-Dreams, Likes, Dislikes, Pleasant and Unpleasant Memories: A 
Study by the Interview Method of 400 Children, Ages 5 to 12. Child Development 
Monographs, No. 12. New York: Teachers College, Columbia University, 1933. Pp. 172. 

3 Ernest R. Mowrer, Domestic Discord, Chapter V, “The Interview Process,” pp. 
53-64. Chicago: University uf Chicago Press, 1928. Pp. 277. 

Gilbert Van T. Hamilton, A Research in Marriage, New York: Albert and Charles 
Boni, 1929. Pp. 570. 


4 Stuart A. Queen, “Social Interaction in the Interview: An Experiment,” Social Forces, 
VI (June, 1028). 545-58. 
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tunity is particularly important in the “treatment inter- 
view” =— used extensively in case work. In other words, the 
interview permits an exchange of ideas and information; it 
is not necessarily a one-way method. 

Employment interviews. Most people who are now em- 
ployed participated in an interview when they were being 
considered for their positions. In an employment interview ° 
certain facts concerning the individual’s training and expe- 
rience are usually sought, although these could very well be 
gathered on application blanks. The principal purpose of the 
face-to-face contact is to afford the employer an opportunity 
to observe the reactions of the respondent, to get some basis 
for judging the quality of the individual’s mental reactions 
and the social aspects of his behavior. Employment interviews 
do not themselves constitute research, but they may be made 
the object of research.” 

Interviews in education and personnel work. Perhaps no 
research technique is as close to the teacher’s vital problems 
as the interview. Not only is it important for research pur- 
poses, but it can be used directly by the teacher in con- 
nection with her everyday classroom problems. It is in fact 
indispensable in diagnosis.° Brownell +° was one of the early 


5 Percival M, Symonds, Diagnosing Personality and Conduct, Chapter XII, “Inter- 
viewing,” pp. 450-84. New York: The Century Co., 1931. Pp. 602. Includes seventy-six 
references in the bibliography. 

Ernest R. Mowrer, op. cit., Chapters IX-XIV, pp. 122-236. 

6 Donald A. Laird, The Psychology oj Selecting Men, Chapter VI, “The Interview,” 
pp. ror-15. (Second edition.) New York: McGraw-Hill Book Co., 1927. Pp. 345. 

‘Technics in Selection and Placement,” p. 282, in “Teacher Personnel,” Review of 
Educational Research, IV (June, 1934), No. 3. 

7 Richard S. Uhrbrock, “Analysis of Employment Interviews,” Personnel Journal, XII 
(August, 1933), 98-101. See also studies referred to in the preceding footnote. 

8 Ruth Strang, The Rôle of the Teacher in Personnel Work, especially Chapter XV, 
“The Interview in Personnel Work,” pp. 325-51, and pp. 401-2. (Revised and enlarged 
edition.) New York: Teachers College, Columbia University, 1935. Pp. 417. 

George W. Hartmann, “Interview as a Research and Teaching Device,” Journal of 
Applied Psychology, XVII (April, 1933), 205-11. 

9 Lofton V. Burge, “Interview Technique as a Means of Diagnosing,” Journal of 
educational Research, XXVII (February, 1934), 422-9. 

10 William A. Brownell, “An Experimental Study of the Development of Number 
Ideas in the Primary Grades.” Doctor’s thesis. Chicago: University of Chicago, 1926. 
Pp. 384. Also published as The Development of Children’s Number Ideas in the Primary 
Grades, Supplementary Educational Monographs, No. 35. Chicago: University’ of Chicago, 
1928. Pp. 241. 
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workers to recognize this fact. Buswell and John™ used 
the technique to discover difficulties in arithmetic, and they 
have incorporated it into their individual diagnostic tests. 
The procedure has proved revealing. Burton ** employed the 
interview in addition to a test to learn what degree of under- 
standing the test results represented. The interview is often 
the only means by which the teacher can discover the detailed 
processes that the child is using in his thinking. It is also 
useful in studying the more general habits of work and ™ 
usually reveals conditions that the average teacher “would 
never have dreamed of.” +5 

The complete pattern of diagnostic research will be pre- 
sented in Chapter X; here attention is called to those analyses 
which can be made directly through interviewing. To illus- 
trate one application, let us assume that a certain fifth-grade 
teacher has a pupil who is having difficulty with arithmetic. 
The teacher has drilled him in the usual fashion; in fact she 
has given him more than the usual amount of drill, but he 
does not improve. His daily work is poor, even though the 
teacher marks his problems with a big red cross and assigns 
extra work for him to do at home. She has given him all 
sorts of arithmetic tests, and his scores are invariably low. 
Thus, having satisfied her teaching conscience by giving the 
pupil extra work, the teacher finally excuses herself by using 
standard tests to “prove” that the pupil simply cannot do 
arithmetic. Instances of this nature occur every semester in 


11 Guy T. Buswell and Lenore John, Diagnostic Studies in Arithmetic. Supplementary 
Educational Monographs, No. 30, Chicago: University of Chicago, 1926. Pp, 212. 

12 Guy T. Buswell and Lenore John, Buswell-John Diagnostic Tests for the Funda- 
mental Processes in Arithmetic. Bloomington, Ill.: Public School Publishing Co., 1926, 

18 William H. Burton, “The Nature and Amount of Civic Information Possessed by 
Chicago Children of Sixth-Grade Level.” Doctor’s thesis, Chicago: University of Chicago, 
1924. Pp. 262, See also his article, “A Contribution to the Technique of Constructing 
Best-Answer Tests,” Elementary-School Journal, XXV (June, 1925), 762-70. 

14 Jessie Allen Charters, “Methods of Study Used by College Women,” Journal of 
Educational Research, X (December, 1924), 344-55. 

Maxie N. Woodring and Zaida F. Metcalfe, “The Use of the Interview in the 
Improvement of Study,” Teachers College Record, XXXV (March, 1924), 480-92, 

15 William A. Brownell and Others, “Remedial Cases in Arithmetic,” Peabody Journal 
of Education, VII (September and November, 1929, January, March, and May, 1930), 
Nos. 2, 3, 4, 5, and 6. A series of five articles dealing with diagnosis and instruction of 
individual childrer- 
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thousands of schools. Yet if this teacher would avail herself 
of a little reading in diagnostic literature which has been 
produced as a result of research by the interview and other 
techniques, she might be stimulated to work with this pupil 
and talk with him about his processes and his difficulties. 
She might find that he was having difficulty because he had 
not learned the basic number combinations of the funda- 
mental operations; or he might not know how to carry when 
adding, multiplying, subtracting, or dividing; he might not 
understand what multiplication really means, or how to start 
division; numbers might have no practical significance to 
him; or he might be handicapped by any number of other 
possible difficulties which are relatively specific and are remedi- 
able. The same type of situation may exist with reference to 
any of the school subjects, and it is likely to continue to 
exist until the teacher or somebody else talks with the pupil 
sympathetically, patiently, and for the purpose of learning. 
Teaching may begin after thoughtful consideration of what 
the teacher has learned from the interview. It is a wise teacher 
who knows when to teach—and when to learn. 

The technique here described, together with the observa- 
tional, experimental, and diagnostic procedures discussed later, 
throws much light on instructional problems. Many teachers 
believe that children do not succeed in school because they 
are mentally below average. Intelligence is one factor in 
school work, but there has been too general a disposition 
for teachers to use it as an excuse. It is every teacher’s own 
responsibility to ascertain the real causes of her pupils’ diffi- 
culties, and not to generalize at the outset that a pupil is 
failing because he is not bright enough, because he is not 
interested in the work, is inattentive, lazy, or absent too much. 
Such types of thinking are too superficial to be justified, with 
the current status of scientific understanding in education. 
Research of the type included in the references cited will 
disclose the real causes of pupil difficulty, including lack of 
interest, absence, and other symptoms of maladjustment. 


382 EDUCATIONAL RESEARCH 


Even though a teacher may not care to engage in producing 
research, she is under obligation as a teacher to keep in touch 
with the current findings of research and to expand her in- 
sights in keeping with the ever growing revelations of research 
on instructional problems. A further obligation of a teacher 
is to make practical application of these insights through 
studying individual pupils. In so doing, she may not be per- 
forming research, but she is meeting her opportunity as a 
teacher by utilizing methods which research has tested and 
found to be valuable. 

In the interest of enlightened teaching it may further be 
pointed out that there are probably a fair number of uni- 
versities with sufficient interest in practical research to ac- 
cept Masters’ theses based upon systematic, properly conducted 
study of the problems in a teacher’s own classroom. Such 
opportunities should be welcomed by teachers. 

School workers other than teachers have many occasions 
for interviewing. It is, in fact, a part of the regular work of 
vocational counsellors,’® attendance officers,’ visiting teach- 
ers, and deans.’® It is of course an essential part of the ad- 
ministration of the Stanford-Binet individual intelligence test, 
and D. S. Snedden*°? has made an informal disguise of an 
intelligence test which is entirely in the form of an interview. 
Although routine uses of the interview are not to be con- 
sidered research, it is important to note practical applications 
of research techniques. Also, such applications may readily 
become research if the worker will integrate his experiences 

‘ into large, embracing observations which grow out of his 

16 Ethel P, Andrus, “Counselor's Interview to Assist in the Choice of Subjects and 
Occupations,” Occupations, XII, Part II (March, 1934), 78-84. 

a Arch O. Heck, Administration of Pupil Personnel, Chapter V, “The Attendance 
Department,” pp. 85-105. Boston: Ginn and Co., 1929. Pp. 479. 

18 Esther Heath, The Approach to the Parent. New York: Commonwealth Fund, 1933. 
TES Culbert, The Visiting Teacher at Work. New York: Commonwealth Fund, 
1929. Pp. 235. 

19 Esther A. Gaw, “Technique Used in the Office of a Dean of Women,” Educational 
Research Bulletin, IX (May 28, 1930), 280-323. 

20 Donald Scott Snedden, A Study of Disguised Intelligence Tests (Interview Form). 


Teachers College Contributions to Education, No. 291, New York: Teachers College, 
Columbia University. 1028. Pp. 48. 
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regular work and afford substantiated insights which increase 
understanding in this field. 

The interview is prominent in social case study ** and in 
psychiatric work.?? The preparation of case histories ** has be- 
come a well-established technique,** involving (where possible 
and appropriate) measurements of intelligence, of educational 
status, of physical condition, and of personality, in addition 
to many data concerning the history of the individual and of 
his family.” In the clinic the psychiatrist will not only obtain 
facts of history from the patient, but will also gain some of 
his more important insights into the case by drawing out 
various comments from the individual. Symonds says, in fact, 
that “The personal interview, then, becomes the most im- 
portant single device available in the differential diagnosis of 
mental disorder.” 2° The nature of case study work will re- 
ceive more complete description in Chapter X. These pro- 
cedures are not foreign to the work of the practical educator ; 
in fact teachers and administrative officers should regularly 
be active in the preparation and use of case histories.” 

Field uses. The interview is the chief method of gathering 
data in the field. Censuses and other similar undertakings de- 


21 Pauline V. Young, Interviewing in Social Work. New York: McGraw-Hill Book Co., 
7935. Pp. 416. 

Porter R. Lee and Marion E. Kenworthy, Mental Hygiene and Social Work, Chapters 
II, TII, and Appendix C. New York: Commonwealth Fund, 1929. Pp. 320. 1931 edition 
has index. 

George A. Lundberg, Social Research, Chapter VIII, “Case Studies and the Statistical 
Method,” pp. 168-96, and pp. 338-41. New York: Longmans, Green and Co., 1929. 
Pp. 380. 

22 Victor V. Anderson and W. M. Kennedy, Psychiatry in Education, New York: 
Harper and Bros., 1932. Pp. 430. See also Lee and Kenworthy, op. cit. 

Percival M. Symonds, “The Province of Psychological Counseling,” Teachers Col- 
lege Record, XXXVI (January, 1936), 313-27. 

23 Caroline B. Zachry, Personality Adjustments of School Children. New York: Charles 
Scribner’s Sons, 1929. Pp. 306. 

24 William Claude Reavis, Pupil Adjustment in Junior and Senior High Schools, 
Chapters III-VI, pp. 41-125. Boston: D. C. Heath and Co., 1926. Pp, 348. 

25 “Interviews, Interviewers and Interviewing in Social Case Work.” Fifteen articles 
reported from The Family. Family Welfare Association of America, 130 E. 22d St., 
New York, 1931. Pp. 133. 

26 Percival M. Symonds, Psychological Diagnosis in Social Adjustment, p. 70. New 
York: American Book Co., 1934. Pp. 362. 

27 Ruth Strang, The Réle of the Teacher in Personnel Work, Chapter XIV, “The 
Technic of the Case Study,” pp. 289-324, and p. 401. (Revised and enlarged edition.) 
New York: Teachers College, Columbia University, 1935. Pp. 417- 
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pend upon the interview. Census takers ordinarily gather 
information concerning the entire family by talking to a single 
representative of the family. Censuses are of course useful in 
education, since they afford the basis for the administration 
of compulsory attendance laws.’ Investigations of the social 
and economic status of the families from which school children 
come% usually involve visiting homes and interviewing par- 
ents,” although information is sometimes obtained at school 
from the children. Studies of home environment will undoubt- 
edly become increasingly common as educators tend more and 
more to recognize the rôles that such factors play in deter- 
mining the needs of school children, and consequently the 
kind of curriculum that they should be pursuing. 

Sociologists and economists gather data for many important 
investigations by means of the field interview.’ Standards 
of living and their costs are amenable to such methods. For 
example, one of our large automobile manufacturers desired 
to establish a rate of pay for its workers in foreign branches 
which would enable them to maintain the same standard of 
living as its employees in this country enjoyed. To do this, a 
careful study ** was made of the purchases in this country by 


28 Arch O. Heck, Administration of Pupil Personnel, Chapters VIII, ‘The School 
Census; Current Practice,” and IX, “The School Census: A Program,” Boston: Ginn 
and Co., 1929. Pp. 479. 

29J. B. Maller, “Economic and Social Correlatives of the School Program in New 
York City,” Teachers College Record, XXXIV (May, 1933), 655-70. 

30 P. M. Symonds, Diagnosing Personality and Conduct, Chapter XV, ‘Measures of 
the Environment.” New York: The Century Co., 1931. Pp. 602. Gives bibliography of 
thirty references. 

Seven scales for rating home and neighborhood conditions are described in some detail 
by Symonds in his Psychological Diagnosis in Social Adjustment, pp. 328-32. New York: 
American Book Co., 1934. Pp. 362. 

See also a review of studies on rating the home, together with the scales which are 
available, p. 207, in Goodwin B. Watson, “Tests of Personality and Character,” Review 
of Educational Research, II (June, 1932), No. 3. 

31 Bruce V Moore, “The Interview in Social Industrial Research,” Social Forces, VIL 
(March, 1929), 445-52. 

George A. Lundberg, Social Research, Chapter VII, “Field Work: The Interview and 
the Social Survey,” pp. 140-67, 336-8. New York: Longmans, Green and Co., 1929. Pp. 
380. 

Robert S. Lynd and Helen Merrell Lynd, Middletown: A Study in Contemporary 
American Culture, pp. 507-9. New York: Harcourt, Brace and Co., 1929. Pp. 550. 

32 “Standard of Living of Employees of the Ford Motor Company in Detroit,” 
Monthly Labor Review of the U. S. Bureau of Labor Statistics, XXX (June, 1930), 
41-54 (or pn. 1200-52.. 
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typical families dependent on the company wages ; correspond- 
ing articles and services were then priced in the foreign mar- 
kets, and the cost of a comparable standard of living 
determined. In order to gather the basic data, trained inter- 
viewers spent a great deal of time in each home. On the basis 
of these interviews they recorded information on a prepared 
list containing 480 items! 

Twice, a similar method has been employed to determine 
the salaries of certain Federal employees.** The effects of the 
depression and consequent unemployment on home life have 
been studied by the interview method.** For some years in- 
dustrial and commercial concerns have analyzed buying prefer- 
ences è of householders by canvassing homes with carefully 
tested questions.** 

Interviewing for analysis and appraisal. Charters * has in- 
dicated a vast field of information relating to the solving of 
practical problems, which can be tapped through the interview. 
That is, persons who are notably successful in their various 
undertakings have certain methods of work, they are guided 
by certain standards, and they react in certain ways, which 
make their work more successful than average. The factors 
that make for good work constitute important information for 
the training of others. By ingenious interviewing—and perhaps 
some observing—one can secure essential differences in the 
thinking and procedures of successful and unsuccessful workers, 
In this way Nietz* interviewed 111 outstanding men in 


33 “Cost of Living of Federal Employees in Five Cities,” Monthly Labor Review of the 
U. S. Bureau of Labor Statistics, XXIX (August—November, 1929), 315-35, 748-59, 
1013-26, and 1039-48. (Available also as a reprint.) See also, “Changes in the Cost of 
Living of Federal Employees in Washington,” pp. 59-60, in “Retail Prices and Cost 
of Living,” for December, 1933. 

$4 Earle E. Eubank, “Consequences of Unemployment in Cincinnati, 1932” in Report 
of the Ohio Commission on Unemployment Insurance, Part II, Studies and Reports, 
Chapter III, pp. 41-69. Columbus, Ohio: The State House, January, 1933. Pp, 304. 

35 Henry C. Link and Irving Lorge, “The Psychological Sales Barometer,” Harvard 
Business Review, XIII (January, 1935), 193-204. 

36 For an index to this literature, see the Psychological Index, Division VIII, Topic 4: 
“Advertising and Selling.” 

37 W. W. Charters, “The Collecting of Unrecorded Specifics,” Journal of Educational 
Research, V (April, 1922), 280-94. 

38 John A. Nietz, “Duties and Traits of a Good Citizen,” Chapter VII, pp. 54-68, 
in Franklin Bobbitt and Others, Curriculum Investigations. Supplementary Educational 
Monographs, No. 31. Chicago: University of Chicago, June, 1926. Pp. 204. 
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Chicago to secure their ideas of the things the average citizen 
should do if he were to perform his citizenship duties well. 

Interviews may play an important rôle in the preparation 
of questionnaires and check-lists which are to be put to ex- 
tensive or crucial use. Reference was made in the preceding 
chapter to the study by Charters and Whitley of secretarial 
duties and traits.*® To secure a set of activities which could 
be used as a check-list, 125 secretaries were interviewed, This 
procedure yielded 871 different duties, and these were sub- 
mitted in check-list form to 2,000 secretaries to obtain the 
relative frequency with which the activities were performed. 
The second part of this study was concerned with discovering 
the relative importance of various vocational and personality 
traits in secretaries, as judged by their employers. For this 
part of the study the investigators depended entirely on the 
interview. 

Employers were induced to talk about the qualities in sec- 
retaries which pleased them, which irritated them, the qualities 
which they would like to improve in their present secretaries, 
etc. These interviews were taken down in shorthand, and the 
typewritten reports analyzed. This procedure probably gave a 
much more valid return than would have been possible if em- 
ployers had hurriedly checked a list, for the mere presence of a 
printed list of traits would to a certain extent spoil the perspec- 
tive of the one responding to it and would result in a different 
order of ranking from that which he would give if analyzing his 
own experience without a list. 

A further difficulty in the use of a list for this purpose is the 
inevitable difference in the interpretation of the terms used for 
the personality or action traits. This would be an additional 
cause of confusion for the person responding, since he would 
feel that many terms in the list overlapped, and he would not 
know which one to check. As a matter of fact, this same ele- 
ment must necessarily be a cause of confusion to the research 


39 W. W. Charters and Isadore B. Whitley, Analysis of Secretarial Duties and Traits. 
Baltimore: Williams and Wilkins Co., 1924. Py. 186. 
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worker in the interpretation of his interview responses ; he must 
strive to put together the different terms used by different 
respondents for the same concepts, or else he will simply pass 
on to the reader the problem of unravelling the confusion. In 
the example referred to in the foregoing paragraph, it was pos- 
sible to ask the employer just what he meant by each term— 
could he give a few illustrations? This is one of the peculiar 
advantages of the interview—following up any lead that is 
given, and stimulating the person to talk on it and develop it. 

The same technique was used by Charters and Waples in 
the Commonwealth Teacher-Training Study.“ Ninety-seven 
interviews were held with administrators, teachers, parents, 
pupils, professors of education, and teachers’ agencies, 


_.. to discover what they believed to be the traits most essential. for 
success in teaching.... They explained the meaning of each trait by 
giving examples of activities whereby the trait was expressed. In this 
way were secured approximately 2,800 actions indicating the possession 
of traits. (pp. 15-16.) 


In this way a master list was secured which was subse- 
quently arranged under twenty-five heads as “traits.” Charters 
points out that the interview was used because 


_ it is possible to secure a much more definite and detailed descrip- 
tion of the traits by the interview, which permits the asking of ques- 
tions. . . . Only by asking a large number of specific questions can 
the interviewer be sure that he has clearly understood and accurately 
recorded the judge’s definition of a given trait. . . . It is essential that 
the terms used in defining a trait be made explicit by synonyms and 
concrete examples. (p. 52.) 


Articles suggesting other applications of the interview to re- 
search will be found in the Education Index under the topic, 
“Interviews.” 

Waples and Tyler show that the data which can be obtained 
through the process of interview are subject to four important 
limitations relating to the respondent: (1) his experience, (2) 


40 W. W. Charters and Douglas Waples, Commonwealth Teacher-Training Study. 
Chicago: University of Chicago Prese- 1927 Po 646. 
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his judgment, (3) his accessibility and willingness to divulge 
the information, and (4) his ability to express himself clearly.** 
The condition of each of these factors should be known before 
data are relied upon. Rice ** has called attention to another 
source of danger in the interview—that of the interviewer 
transferring to the respondent some of his own ideas or lean- 
ings. Most people realize how much one tends to agree with 
another person in conversation on inconsequential topics. The 
one being interviewed may have little idea of the significance 
of the things he is saying and hence will in many cases tend to 
respond in accordance with any suggestion of an attitude or 
general trend that is dropped by the interviewer. 

Preparation for the interview. Interviewing is an art and 
requires long study and experience for its development. Some 
guidance can be obtained from printed material, but one must 
expect to undergo a considerable period of practice before he 
is able to obtain through the interview the results which it is 
capable of yielding. The limited suggestions given here will 
serve mainly to introduce the literature and to emphasize the 
necessity for appropriate planning. 

As in preparing the questionnaire and other schedules, one 
must give considerable thought to the planning of questions 
he is going to ask. An interview for research purposes cannot 
be a haphazard affair, or just a pleasant meeting. These state- 
ments do not mean that the interview should proceed simply 
as a series of questions. It should be pleasant and to some ex- 
tent informal, but underneath this social naturalness must be 
a thread of questions that will give to the interviewer the 
information he is seeking, without gaps and without doubt- 
ful interpretations. 

The interviewer will normally have a set of carefully pre- 
pared questions ** which he will introduce into the conversa- 


41 Douglas Waples and Ralph W. Tyler, Research Methods and Teachers’ Problems, 
p. 520. New York: The Macmillan Co., 1930. Pp. 653. 

42 Stuart A. Rice, “Contagious Bias in the Interview: A Methodological Note,” 
American Journal of Sociology, XXXV (November, 1929), 420-3. 

43 Several studies cited in the foregoing discussion of the applications of the interview 
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tion at appropriate points. He may vary such queries to adapt 
them to individual circumstances; he will usually amplify 
questions by following up leads as the occasion may suggest, 
since this is the great opportunity of the interview. To avoid 
a meaningless set of material when he has his facts all gath- 
ered, however, all of the interviewer's work must be done with 
a plan. He must have decided in advance, or before going 
very far, just what information is necessary to draw conclu- 
sions that will satisfy his principal purpose in making the 
study. 

The interview provides unlimited opportunity for the use 
of tact. Some information can be secured by direct ques- 
tions; much more can be secured indirectly. This is because 
the respondent recoils more or less from the thought of re- 
vealing certain information too openly and bluntly and does 
not generally know the significance of certain details which 
he omits mentioning. He probably has not been thinking about 
the material from the angle of the investigator. Again, in the 
case of children, they do not have sufficiently mature concepts 
to give a complete recount of experiences or ideas.* The re- 
sponsibility rests upon the interviewer to sample rather thor- 
oughly the knowledge and attitudes of the person being 
interviewed, through skilfully drawing out responses that rep- 
resent various elements in the person’s thinking. This is where 
tact and technique count. 

Symonds’ treatment *° includes within a brief compass the 
more important aspects of interviewing, such as the preview, 
the initial contact, conditions of the interview, securing rap- 
port, “rising action,” lessening tension, etc. He gives a bibliog- 
raphy of seventy-six references, including most of the impor- 
tant ones prior to the publication of his book. His analysis 
affords an excellent brief discussion. For longer, more detailed 


technique contain examples of schedules of questions which have been prepared as a 


basis for the interview. . x i y 
44 “The Psychoanalytic Method,” in “Methods and Techniques of Educational Re- 
search,” p. 70, Review of Educational Research, IV (February, 1934), No. 1. 
45 Percival M. Symonds, Diagnosing Personality and Conduct, Chapter XII, “Inter- 
viewing,” pp. 450-84. New York: The Century Co., 1931. Pp 602. 
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treatises, the reader should consult the standard works of 
Bingham and Moore and of Young.** 

Uhrbrock ** has suggested that stenographic reports furnish 
an important means for the study and improvement of one’s 
interviews. Bixler 48 gives twenty-two points for conducting 
interviews. Lundberg +° treats the problem of interviewing in 
social work, including the supporting administrative organiza- 
tion. Several other references may prove valuable.°° 


5. STUDIES OF BEHAVIOR EMPLOYING OBSERVATIONAL PROCEDURES 


Analysis of documentary material, treated in the preceding 
chapter, is concerned with what people think and do, as re- 
vealed by what they put on paper. The interview, just dis- 
cussed, reveals what people think and do by what they express 
in conversation with the interviewer. Observation seeks to 
ascertain what people think and do by watching them as they 
express themselves in various situations, selected to typify 
the conditions of informal living or to represent some special 
set of factors. 

Observation is recognized at once as the most direct means 
of studying people when one is interested in their overt be- 
havior. In an interview people may tell what they think they 


46 Walter Van Dyke Bingham and Bruce Victor Moore, How to Interview. (Revised 
edition.) New York: Harper and Bros., 1934. Pp. 308. 

Pauline V. Young, Interviewing in Social Work. New York: McGraw-Hill Book Co., 
1935. Pp, 416. 

47 Richard S. Uhrbrock, “Analysis of Employment Interviews,” Personnel Journal, 
XII (August, 1933), 98-101. 

48H. H. Bixler, Check Lists for Educational Research, pp. 38-40. New York: 
Teachers College, Columbia University, 1928. Pp. 118. 

49 George A. Lundberg, Social Research, Chapter VII, “Field Work: The Interview 
and the Social Survey,” pp. 140-67, 336-8. New York: Longmans, Green and Co.. 
1929. Pp. 380. 

50H. L. Myrick, Interviews: A Study of the Methods of Analyzing and Recording 
Social Case Work Interviews. Studies in the Practice of Social Work, No. r. American 
Association of Social Workers, 1928. Pp. 75. 

Sarah M. Sturtevant and Harriet Hayes, “The Use of the Interview in Advisory Work,” 
Teachers College Record, XXVIII (February, 1927), 551-62. 

Additional references will be found by consulting “Interview,” p. 38 of the Subject 
Index, in: W. H. Cowley, The Personnel Bibliographical Index, Columbus: Bureau of 
Educational Research, Ohio State University, 1932. Pp. 433. 

A, S. Barr and Mabel Rudisill, An Annotated Bibliography on the Methodology of 
Scientific Research as Applied to Education, pp. 87-93. Bulletin of the Bureau of 
Educational Research, No. 13. Madison: University of Wisconsin, June, 1931. Pp. 129. 


—_— 


THE NORMATIVE-SURVEY METHOD 391 


do, but this is often different from what they really do, for 
they are not generally accurate »bservers of themselves. 
Furthermore, they may not wish to give accurate descriptions 
of themselves; few of us would care to, if we could. It is true 
that the interview is itself a form of expression, and one may 
be interested in studying the behavior of others when they 
are being interviewed; in that case, however, it is really obser- 
vation that one is depending upon, since the interview then 
becomes simply a means of setting up a given situation under 
which to observe people. 

Direct observation of behavior has only recently come to be 
looked upon as a scientific procedure; in fact, a great many 
scientists would at the present time express serious misgivings 
as to the possibility of anything “objective” coming from such 
a method. Regardless of questions about its scientific nature, 
the fact remains that there are many aspects of behavior which 
can be studied in no other way, and one has to choose between 
bearing the criticism of those who emphasize the formalities 
of science, and foregoing the study of many characteristics 
which are of large practical importance, It would seem reason- 
able that the first requirement of a research technique is that 
it be adapted to the kind of data and the field conditions with 
which it must deal. Other criteria of science are important, 
but secondary. 

Recent changes and use. In recent years there has been an 
increasing number of studies of infants and older children, 
concerned with various activities and characteristics, and based 
upon direct observation." The rapidly growing interest in child 


51 For current references to studies based on observation, consult the Education Index 
for the topic “Behavior” and “Child Care”; consult the index of the annual United 
States Office of Education Bibliography of Research Studies in Education for the topics 
“Behavior,” “Child Study,” and ‘Preschool Education.” In the Psychological Index, 
studies utilizing observation are likely to be found under the following heads: VI—1, 
“Attention and Interest”; VII, “Social Functions of the Individual”; XI, “Mental 
Development in Man.” 

A review of studies based upon observation will probably be found in the 
Psychological Bulletin of July each year, where various tests are reviewed. For example, 
see the comments by Maller in the Psychological Bulletin, XXXII (July, 1935), 510. 
See footnote 32 of Chapter VII for further details of this series. 

‘Another review of observational studies is given by George D- Stoddard, “Observational 
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study has shifted attention to a great variety of aspects of 
child growth, and some method of measuring the status of 
these different characteristics has become necessary. Attempts 
to measure attainment of the objectives of progressive educa- 
tion have furthered this work. Supervision of classroom teach- 
ing depends in large part upon observational methods. 

At first, the data that were gathered tended to be too in- 
clusive. Stoddard comments: * 


In the past such records consisted of omnibus accounts of every- 
thing the child did or said over long periods of minute observation. 
Voluminous data resulted which were unique for the child being ob- 
served. These sometimes led to improved insights on the part of the 
person doing the recording, but they did not contribute much to the 
working out of a scientific approach. 


As the technique of direct observation has been refined, the 
tendency has been to restrict the data to a limited (although 
possibly very complex) characteristic of behavior which could 
be identified readily by one trained in the work. The results 
of such observation tend to become measurements of a single 
trait and can be studied as such. The older case-history types 
of data, however valuable they might be for clinical use with 
an individual child, were not readily amenable to statistical 
treatment. Further, as Stoddard states, “Each set of material 
contains so much of the peculiar ability, attitude, or even 
prejudice of the person who assembled it.” ** By limiting the 
scope of the data to be gathered and by careful definition of 
the characteristic to be observed, the newer attempts seek to 
eliminate much of the personal variation in the observer.** The 


Method,” pp. 65-6, in “Methods and Techniques of Educational Research,” Review of 
Educational Research, IV (February, 1934), No, 1. See also the discussion in George D. 
Stoddard and Beth L. Wellman, Child Psychology. New York: The Macmillan Co., 1934. 
Pp. 419. 

Percival M. Symonds devotes a chapter to observation (Chapter II) in his Diagnosing 
Personality and Conduct. New York: The Century Co., 1931. Pp. 602. 

A number of annotated references will be found in: A. S. Barr and Mabel Rudisill, 
An Annotated Bibliography on the Methodology of Scientific Research as Applied to 
Education, pp. 83-6. Bulletin of the Bureau of Educational Research, No. 13. Madison, 
Wis.: University of Wisconsin, June, 1931. Pp. 129. 

52 Stoddard, op. cit., p. 65. 

53 Ibid., p. 69. 

54 A tabular comparison of the characteristics of the newer ‘‘systematized’’ observation 
and the earlier more “unsystematized” observation is given by: Florence L. Goodenough 
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reliability of such observation compares favorably with the 
reliability of paper-and-pencil tests.** 

As a concrete application of this point of view, a series of 
studies have been made in the Child Development Institute of 
Teachers College, Columbia University, under the general di- 
rection of Dorothy Swaine Thomas. These are known as the 
Child Development Monographs," of which more than twenty 
have been published to date. In the first volume,” published in 
1929, Thomas states that the newer techniques of observation 
which are used in this series are designed to overcome the 
difficulties of the older techniques—particularly of the diaries 
and case studies. She points out that when one is observing 
behavior in general, he will notice and record different elements 
at different times, and that the records tend to be a mixture 
of fact and interpretation because the traits have not been well 
defined in concrete terms. To overcome these difficulties, only 
specific reactions of the child are noted. Each study has dealt 
with a characteristic selected on the basis of its judged signifi- 
cance in the total behavior patterns of individuals. 

In explaining why she does not regard the controlled experi- 
ment as satisfactory, Thomas states that the real problem is to 
note the particular stimuli (out of “the multifarious stimuli 
which comprise their social milieu”) to which individual chil- 


and John E. Anderson, Experimental Child Study, pp. 428-9. New York: The Century 
Co., 1931. Pp. 546. 

A description of important steps in the development of present techniques is given in: 

Dorothy Swaine Thomas and Others, Observational Studies of Social Behavior, Vol. I, 
“Social Behavior Patterns.” New Haven, Conn.: Institute of Human Relations, Yale 
University, 1933. Pp. 271. See especially Chapter I, pp. 1-42. 
The successive steps in the improvement of observational procedure in classroom 
supervision are given by A. S. Barr, An Introduction to the Scientific Study of Class- 
room Supervision, pp. 4-5, and Chapter V. New York: D. Appleton and Co., 1931. 
Pp. 399. 

55 Symonds, op, cit., pp. 33-6. 

See also Thomas and Others, op. cit., especially Chapter V, and Part II. 

A. S. Barr, op. cit., especially Chapter V, “The Validation of Data-gathering Devices,” 

. 190-248, 

Bes For ‘bibliographical purposes the latest volume in this series at the date of writing 
is given: Arthur T. Jersild and Sylvia F. Bienstock, Development of Rhythm in Young 
Children. Child Development Monographs, No. 22. New York: Teachers College, 
Columbia University, 1935. Pp. 97- This volume contains a list of the preceding 
twenty~ tudies published in this series. 

s PEET Sa Thomas and Associates, Some New Techniques for Studying Social 
Behavior. Child Development Monographs, No. 1. New York: Teachers College, 


Columbia University, 1929. Pp. 203. 
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dren react, and to study the consistency of this reaction in an 
individual, as well as the variability in such reactions between 
different members of the group. 

Thomas is apparently thinking of the artificiality of many 
experimental situations, especially in laboratories. It should be 
clear, however, that the conditions under which observation 
takes place may approach and actually satisfy the require- 
ments for a carefully controlled experiment. (The general 
nature of experimental research is explained in the following 
chapter, IX.) The general conditions of observation may be 
controlled by selecting the room, the equipment, the children 
who make up the group, special forms of stimuli, and the ob- 
servers. The remaining requirement in experimentation is the 
study of the effects produced by changes in a certain factor. 
A number of investigations which have used direct observation 
under conditions that make them practically experiments have 
appeared in the series of bulletins known as the University of 
Towa Studies in Child Welfare." These investigations have been 
made in the Iowa Child Welfare Research Station which spon- 
sors studies of children covering nutrition, physical growth, 
child psychology, preschool education, and character education. 

The University of Minnesota has also contributed studies of 
the direct observation type. W. C. Olson is credited by one 
writer with making the pioneer adaptation of the observation 
method in a rigorous form to educational work,*® through de- 
vising and validating the “time sampling” technique. Other 
persons at the same university, working in the Institute of 
Child Welfare, have also utilized this method.*° Scattered 
studies of the observation of children have of course been made 
in many institutions throughout the country. Interest in ob- 

58 See for example, Lois M, Jack and Others, “An Experimental Study of Ascendant 
Behavior in Preschool Children,” in Researches in Child Behavior. University of Towa 
Studies in Child Welfare, Vol. IX, No. 2. Iowa City: University of Iowa, 1933. 

59 Williard C. Olson, The Measurement of Nervous Habits in Normal Children. 
{nstitute of Child Welfare Monograph Series, No. II, Minneapolis, Minn.: University 
of Minnesota Press, 1929. Pp. 97. 

60 See the “Annotated Bibliography of the Publications of the Institute of Child 


Welfare, 1925-1934,” (or later issue), Minneapolis, Minn.: University of Minnesota, 
June, 1934. Pp. 82. Contains 267 references. 
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servation for purposes of classroom supervision has been mani- 
fested in a number of centers, as recorded by Barr (see foot- 
note 54). 

Observing large outcomes of education. Direct observation 
of behavior has become an important means of appraising the 
work of progressive schools * and of teachers who are in- 
terested in certain outcomes in addition to the academic ones. 
Many of the larger goals of education fall outside of the range 
of the usual academic abilities, embracing in general the social 
characteristics of the individual (such as coöperation and 
adaptability), the self-reliant traits (appropriate independence 
and self-confidence), the dynamic traits (initiative and spon- 
taneity), good judgment in choosing companions, and other 
elements of the well-rounded, well-developed individual. Such 
characteristics cannot be indexed satisfactorily by the methods 
of testing which have heretofore been used.” One may, of 
course, make a direct appraisal of the work of a class by just 
“looking on,” but a more reliable procedure is to look for 
definite acts which are regarded as evidence of the attainment 
of certain objectives.’ After the observation, the frequency 
with which these significant acts have occurred may be taken 
as an index of the extent to which certain habits have become 
established. Groups which exihibit these significant acts with 
greater or less frequency show more or less evidence of having 
grown in the desired direction. 

Caswell and Campbell point out that evaluation is in 
reality a part of the curriculum, and as such it is being given 


61 J. Wayne Wrightstone, “Measuring Intangibles in Progressive Schools,” Progressive 
Education, XII (February, 1935), 95-7. 

A. S. Barr, An Introduction to the Scientific Study of Classroom Supervision, Chapter 
III, “The Survey of the Products of Learning,” pp. 62-115. New York: D. Appleton and 
Co., 1931. Pp. 399. 

62 Ralph W. Tyler, ‘Techniques for Evaluating Behavior,” Educational Research 
Bulletin, XIII (January 17, 1934), I-11. 

63 Frederick Pistor, ‘Valid Scientific Appraisal of an Enterprise in Progressive Educa- 
tion,” Journal of Educational Research, XXVIII (February, 1935), 433-49- 

64 Consult the Education Index for appropriate articles on “Progressive Education,” 
and see other articles listed by such writers as Wrightstone, Tyler, and Pistor. 

65 Hollis L. Caswell and Doak S. Campbell, Curriculum Development, Chapter XIII, 
“Evaluating the Outcomes of Instruction,” pp. 362-76. New York: American Book Co., 


1935. Pp. 600. 
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definite attention in modern courses of study. After a brief 
discussion of the inadequacies of the more common methods 
of measurement, in the light of important purposes which 
evaluation should serve, they refer to the appraisal of “gen- 
eralized controls of conduct.” Citing “The Course of Study for 
Florida Elementary Schools, Grades I to VI,” 1933, pp. 766-70, 
they quote: “Observational tests are much more important 
than teachers usually realize. More can be told about the com- 
plete development of a child by observing him from day to 
day than in any other way.” The course of study recommends 
observing the pupils to obtain answers to such questions as: 
“Does he work well with a group?” “Is he on time?” “Is he 
pleasant and cheerful?” “Does he read good books during 
leisure time?” etc. The observational technique is also recom- 
mended for individual subject fields, to cover such general 
traits as self-reliance, responsibility for personal growth, in- 
terest, etc. 

A different form of observation is represented by the 
anecdotal technique. This consists in writing down each day 
certain revealing sayings or doings of children. A teacher may 
note some half dozen of these a day, for various children. 
Over a long period of time, such records give a significant pic- 
ture of a child’s development."* It is obvious that this type of 
observation and recording embraces situations which are as 
natural as possible in the classroom or school environment, and 
that it is free from strictures imposed by a prepared record 
form or a specially controlled situation. Therefore it depends 
entirely on the selective ability of the teacher. When well kept, 
however, such records provide an important type of informa- 
tion about children. They have been given extended use in the 
Rochester Athenaeum and Mechanics Institute." 


66 Louis Raths, “Anecdotal Records.” Bulletin No. x of the research staff in the 
Kight-Year Evaluation Experiment of the Progressive Education Association. Columbus, 
Ohio: Bureau of Educational Research, Ohio State University, September, 1935. Pp. 9. 

Ben D. Wood, “Information for Guidance: The Anecdotal Method of Personal 
Analysis,” Occupations, XIII (June, 1935), 795-803. 

67J. A. Randall, “The Anecdotal Behavior Journal,” Progressive Education, XIII 
(January, 1936), 21-6. 
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H. C. Morrison has devised an observation procedure, with 
record blanks, for ascertaining the amount of attention during 
a class period." Knudsen shows at some length how this may 
be used and interpreted.*® He points out that attention, 
although intrinsically a conscious state, has accompanying 
forms of physical expression. Schmidt is reported to be devel- 
oping an observational technique for industrial-arts pupils that 
“goes beyond knowledges and skills into the fundamentals of 
habits, attitudes, and appreciations.” *° 

The appraisal of educational practices, especially when 
check-lists, rating scales, or score cards are used, is treated 
more broadly in the following section on “Appraisal Proce- 
dures,” Other applications will be found in that treatment. 

Special forms of recording. In addition to note-taking dur- 
ing direct observation, there are various special forms of re- 
cording that provide a more complete record of what takes 
place. For example, shorthand records may be made of con- 
versations, still or motion pictures may be taken, sound rec- 
ords may be made, and other devices may be employed. The 
study of records made by such means may be regarded as 
indirect observation, in contrast with the direct observation 
that has been discussed. Although records are of course made 
in the case of direct observation, they are the records of the 
observer’s judgment of what happened; the records referred 
to as leading to indirect observation are sufficiently complete 
so that one can secure from them the information desired in- 
stead of haying to depend directly upon the actual situation. 

Stenographic notes lie between direct observation and me- 
chanical means of recording. The human element of the re- 
corder may enter to a considerable extent. The study of 
shorthand reports is to be regarded, however, as indirect ob- 


68 Henry C. Morrison, The Practice of Teaching in the Secondary School, Chapter VIII, 
“Measuring Group Control,” pp. 121-49, and Chapter IX, “Sustained Application,” 
pp. 141-57. (Revised edition.) Chicago: University of Chicago Press, 1931. Pp. 688. 

69 Charles W. Knudsen, Evaluation and Improvement of Teaching (in secondary 
schools), Chapter VIT, “Evaluation and Improvement of the Teacher's Group Control,’ 
pp. 249-84. Garden City, N. Y.: Doubleday, Doran and Co., Inc., 1932. Pp. 538. 

70 See Review of Educational Research, IV (December, 1934), P. 497- 
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servation. Stenographic records have been made chiefly in the 
field of supervision and teacher training."* At times there has 
been some disposition to draw conclusions from them as though 
they represented the classroom situation in its entirety. Such 
use must prove both misleading and disappointing. It is neces 
sary that the limitations of all research data be recognized, 
and that they be used only for the purpose for which they are 
suited. Stenographic reports of classroom conversation give 
evidence of the relative amounts of verbal participation by 
teacher and by pupils; they show the extent to which participa- 
tion is concentrated or scattered in the class; they reveal the 
type of English used; and they may show the topical emphasis. 
On the other hand, there are many things which such reports 
cannot reflect—the dynamic aspects of the teacher’s person- 
ality, the enthusiasm of the group, the shifting of overt at- 
tention, the tempo of the intellectual exchange, etc., in addi- 
tion to many other non-verbal elements in the classroom situa- 
tion, They do not in themselves afford a satisfactory basis for 
judging the work of a teacher beyond certain restricted aspects. 

The study of oral English has been greatly retarded by the 
lack of suitable means for recording and analyzing. Steno- 
graphic records have been used to some extent, but their ad- 
ministration is difficult, they are costly, and are not entirely 
accurate. A sensitive, electric recording device has been devel- 
oped by Betts, which promises to open up new opportunities in 
this field. An extensive study of oral language“? has been 

71H. A. Peterson, E. A. Turner, and B. C. Moore, Stenographic Reports of Elementary 
School Lessons. (Second edition.) Bloomington, Ill.: Public School Publishing Co., 1929. 
Pp. 96. 

See also the topic, “Stenographic Reports of Lessons,” in the Education Index for 
references to lessons reported from many different fields of work. 

72 For a general description of the work at the University of Iowa, see Harry A. 
Greene, “Research Techniques Peculiar to English,” pp. 133-9, in The Application of 
Research Findings to Current Educational Practices, Official report of 1935 meeting of the 
American Educational Research Association. Washington: The National Educational Asso- 
ciation, 1935. Pp. 273. 

For a report of the validation of an electrical record device, see Emmett A. Betts, 
“An Evaluation of Certain Techniques for the Study of Oral Composition,” pp. 7-32, 
in Research Studies in Elementary School Language, No. 1. University of Towa, Studies 
in Education, IX (March 15, 1934), No. 2, New Series No. 271. Iowa City, Iowa. Pp, 62. 


Bibliography of thirty-four references. 
As an illustration of the study of oral language usage through shorthand reports, see: 
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undertaken at the University of Iowa, under the direction of 
Harry A. Greene, in which electrical equipment will be used 
for recording and Hollerith tabulating equipment for analysis. 
Normative studies of language usage of different groups of chil- 
dren and their errors, genetic studies of the development of 
language patterns, and experimental studies of the effects of 
different teaching procedures, are important fields which may 
be more thoroughly investigated in the future with the aid of 
new mechanical devices. 

An extensive study which utilized both motion pictures and 
stenographic notes was made by Gesell, at Yale University, on 
infants. This large work, eventuating in An Atlas of Infant 
Behavior, was carried on over some twenty-five years in 
which goals were defined and techniques sharpened. Gesell was 
interested in ascertaining what constituted normal behavior of 
infants at various ages, and children’s typical reactions to test 
situations are shown in the first volume of the Atlas. The sec- 
ond volume deals with “naturalistic behavior,” and shows chil- 
dren’s typical reactions in ordinary home situations, such as 
feeding, bathing, playing, etc. Although the study as a whole 
is essentially genetic and therefore will be discussed in Chapter 
X, it has normative aspects insofar as it is directed toward the 
establishing of typical behavior patterns of children at various 
ages. 

Mention should be made here of the use of moving pictures 
in the study of physical activities, such as the movement of 
players of a football team. Such pictures have the advantage 
of being able to present the action slowly so that it can be 
analyzed. Also, each player can see himself from another per- 
spective than his own. Gymnasium performance has been an- 
Mary S. Fisher, Language Patterns of Preschool Children. Child Development Mono- 
graphs, No. 1. New York: Teachers College, Columbia University, 1934. Pp. 88. A 
summary is given in the Journal of Experimental Education, I (December, 1932), 70-85. 

Sa Arnold Gesell and Others, An Atlas of Infant Behavior. New Haven, Conn.: Yale 
University Press, 1934. 2 vols. 
For many purposes the following work will serve as 


Helen Thompson, and Catherine Amatruda, Infant Beha 


New York: McGraw-Hill Book Co., 1934. Pp. 343- — 
A brief and enlightening account of this study is given in Fortune, XI (May, 1935), 


86-9r, under the caption, “Behold the Babe.” 


well or better: Arnold Gesell, 
wior: Its Genesis and Growth, 
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alyzed in the same fashion. One may use such methods for 
gathering data for many types of research problems. 

The recording of certain aspects of behavior by mechanical 
means tends to make the observation objective. The observer 
may look at the same record repeatedly and thus improve his 
judgment as to what actually happened, in contrast with the 
possibility of an immediate judgment based upon a single im- 
pression of something that has ceased to exist after an instant. 
Furthermore, other workers may also study the record, and if 
all agree that the behavior recorded represents a certain kind 
of activity, the method may be said to be highly objective. 

An important question with reference to such methods of 
recording, so far as normative-survey research is concerned, is 
whether they cover at one time enough different elements of 
behavior, of the individual and of the group, so that all of the 
different kinds of significant factors are recorded. It must be 
remembered that the distinctive contribution which direct ob- 
servation can make lies in its capacity for dealing with very 
complex phenomena, especially those occurring in social situa- 
tions. It is quite possible to vitiate this opportunity through 
making the character one observes so narrow and simple that it 
loses the significance it should have for contributing to the 
understanding of behavior in such situations. This result may 
occur whether or not one resorts to mechanical means of re- 
cording, but any objective form of recording must not in itself 
impose restrictions upon the nature of the characteristic which 
it is desired to observe. 

Estimates from photographs. Perhaps an illustration of this 
standpoint is to be found in the difficulty of estimating the in- 
telligence or other personal characteristics of an individual 
from a still picture. One may say at once that “motion” is not 
portrayed by still pictures, but it is equally true that any 
mechanical—or even direct—form of observation may fail to 
record other elements which, though less immediately obvious, 
are just as important for one’s purpose. 

In most of the forms of personality appraisal, direct observa- 
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tion of behavior is an essential. It is usually of the incidental 
kind, forming the background for ratings. It has been found 
also that in giving individual intelligence tests the examiner 
can learn many things about the intelligence and personality 
of the subject by noting various elements in his behavior,™* 
in addition to the responses formally recognized. 

Observation of more static characteristics, however, such as 
facial features, cannot be relied upon to provide a satisfactory 
basis for judging either character or intelligence. After re- 
viewing a number of studies of judging character on the basis 
of looking at an individual or his photographs, Watson 
states: 


The conclusions of all these investigators are well in accord: tem- 
porary emotional states can be identified from facial features, but the 
more permanent trends in character are not indicated by the measure- 
ment of any facial features. Later investigations ... all point in the 
same direction. 


The judging of intelligence from photographs has failed 
almost completely. Phrenology and physiognomy are con- 
demned by scientific evidence.” Thus the observation of rela- 
tively static physiological characteristics of human beings, 
either directly or through photographs, is not to be classed 
with the observation of overt behavior. 


74Harry J. Baker, “Clinical Interpretations,” pp. 189-91 in “Psychological Tests,” 
Review of Educational Research, V (June, 1935), No. 3. 

Janet Fowler Nelson, Personality and Intelligence: A Study of Some Responses Other 
than Intellect Noted in a Simple Mental Test Situation. Child Development Monographs, 
No. 4. New York: Teachers College, Columbia University, 1931. Pp. 62. 

s ‘Goodwin B. Watson, “Character Tests and Their Applications Through 1930,” p. 197, 
in “Teste of Personality and Character,” Review of Educational Research, IX (June, 1932), 
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70 Donald A. Laird, The Psychology of Selecting Men, pp. 109-15. (Second edition.) 
New York: McGraw-Hill Book Co., 1927. Pp. 345. 

Katherine T. Omwake, “The Value of Photographs and Handwriting in Estimating 
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Laboratory observation. In general, laboratory studies in- 
volve mechanical records and may be considered an indirect 
type of observation. Furthermore, they are usually confined to 
a type of behavior which is not immediately social in its stimu- 
lation or expression. It is not our purpose to discuss here the 
various means of securing laboratory records, since these 
means will be referred to in the next chapter which deals with 
experimental research. It is appropriate at this point, however, 
to call attention to the fact that laboratory investigations nor- 
mally rely upon mechanical means of observing behavior. Due 
to this fact, and to the fact that the characteristic being studied 
is usually made very simple, laboratory investigations are fre- 
quently thought of as producing relatively artificial results; 
that is, not typical of usual life situations. 

It is true that the laboratory studies, in the narrow sense, 
are not capable of measuring directly the more complex types 
of response which constitute the domain of direct observation ; 
they may, nevertheless, produce results which have large sig- 
nificance for education. This comes about primarily through 
the discovery of some response which, though fairly simple in 
itself, is indicative of variation in a much more complex re- 
sponse. Two illustrations will serve to clarify this statement. 
For one example, we may look to laboratory studies of eye- 
movements. Although eye-movements in reading are relatively 
simple in themselves, they may be taken as an indication of 
the direction of one’s attention. By photographing eye- 
movemenis, therefore, and noting how many pauses are made 
in reading a line of type, where these pauses are made, what 
their duration is, under what circumstances one makes regres- 
sive movements, etc., a great deal can be learned about the 
difficulties the subject is experiencing in reading. In studying 
eye-movements of a subject looking at a picture, one can learn 
how people explore pictures, what features catch their atten- 
tion, what the artist looks at in a picture, etc. Such studies may 
be normative in character, if made for the purpose of estab- 
lishing prevailing habits, or they may be experiments, if made 
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for the purpose of studying variation in performance which 
follows changes in some factor, such as different sizes of type, 
different lengths of line, different kinds of reading material, 
pictures in different colors, etc. These studies will be reviewed 
at some length in the following chapter. 

As a second illustration, we may consider the experiments 
made by Renshaw and his colleagues in connection with the 
Payne Fund Studies of Motion Pictures and Youth.’* Among 
many experiments conducted in this study, one was concerned 
with the effect of movie attendance upon restlessness during 
sleep. A hypnograph was placed beneath bed springs, to reg- 
ister the number of movements which the subjects made while 
asleep. Many readings were taken first to establish the normal 
amount of movement; this much of the study may be regarded 
as normative. Further readings of restlessness were taken after 
children attended movies of various kinds, and these readings 
were compared with the norm. As a matter of interest, studies 
were made of the effect of other factors, such as drinking 
coffee, staying up late at night, etc. The study of the effects of 
these several factors, one at a time, makes the investigation an 
experiment. The aspect with which we are here concerned, 
however, is that the simple, mechanically recorded character, 
number of movements made during sleep, represents one form 
of physiological response to emotional tension or nervous con- 
dition and, when studied, affords some insight into the effect 
of movie attendance and other special factors. 
_ It will thus be clear that laboratory observation may study 

problems of large moment through recording relatively simple 
forms of behavior. In so doing, it is only extending the method 
used in direct observation, namely, the use of some charac- 
teristic of behavior as an indication of psychological response 
tendencies which underlie the overt activity. Laboratory 


78 Samuel Renshaw, Vernon L, Miller, and Dorothy Marquis, Children's Sleep. New 
York: The Macmillan Co., 1933. Pp. 242. Briefly described in W, W. Charters, Motion 
Pictures and Youth: A Summary, pp. 31-5. New York: The Macmillan Co., 1933. 
Pp. 66. Or see: Samuel Renshaw, “Sleep Motility as an Index of Motion Picture 
Influence,” Journa! of Educational Sociology. VI (December, 1932), 226-30, 
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studies, in the usual sense, are ordinarily concerned with be- 
havior in non-social situations. Mechanically recorded observa- 
tion may, however, be used to study behavior in situations in 
which the predominating stimuli are social, as previously dis- 
cussed, care being taken to see that the means of recording 
portray a sufficiently large or complex aspect of behavior to 
retain elements vital for one’s purpose. Any of the procedures 
which have been presented—direct or indirect observation of 
social behavior, or laboratory observation in its restricted sense 
—may be directed toward normative, experimental, genetic, 
case study, or other purposes. It is the larger frame of a study, 
rather than the techniques employed, which determines its 
methodological character. 

Characteristics of observation for research. In concluding 
the discussion of observation, it may be well to summarize the 
characteristics which distinguish it from ordinary looking 
around and entitle it to the claim of a scientific procedure. 
There are at least six characteristics which may be mentioned 
in this connection. First, the observation is specific. It is not 
just looking around for general impressions (however satisfac- 
tory that may be for non-scientific purposes). There are defi- 
nite things for which one looks. These things, furthermore, 
have been carefully defined, so that there is practically nothing 
left to the judgment of the observer. All questions of whether 
a certain activity constitutes an instance of what one is looking 
for are supposed to be ironed out in the preliminary trials of 
the technique, 

Second, scientific observation of behavior is systematic. It 
is not a chance “dropping in” on a situation at any time when 
one happens to be passing by. The length of the observation 
periods, the interval between them, and the number are care- 
fully planned. These plans are made in the light of other 
factors which are known—for example, the daily program of 
the school the children attend, the visiting hours, the periods 
when the children may be tired, etc. Also, there may be careful 
arrangements for the control of the situation if special factors 
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are to be watched, as when the study takes on some of the 
aspects of an experiment. The entire program of observation 
is planned with a view to meeting the requirements of scientific 
work. 

Third, the observation is quantitative. Usually the number 
of instances of a certain type of behavior are recorded. Some. 
times it is the total duration of the particular conduct during 
the period of observation, or it may be the distance traveled 
by a child, or some other countable or measurable charac- 
teristic. Sometimes a diagram is made showing spatial rela- 
tionships. Such relatively exact qualification is another 
distinguishing feature as compared with everyday observation. 
Although it is implied in the foregoing thought, perhaps it 
is worth mentioning as a separate, and fourth, characteristic 
that a record of the observation is made immediately. The 
results are not entrusted to memory, but notes are made as 
promptly as possible. This is more important than it may 
appear at first thought, and scientists will recognize its sig: 
nificance. 

Fifth, the observation is expert. That is, it is done by some 
one who is trained to do such work. This statement may, as 
a matter of fact, appear hopeful, when one considers the many 
studies which are made by students who are just beginning. 
Such studies, however, are not likely to be the ones on which 
the science of education ultimately rests, and it is fair to say 
that the fundamental investigations in education will be made 
by those who have had ample training in the appropriate 
techniques. The difference between the observation of one whe 
is trained and one who is not trained, both in the use of the 
technique and in the field for which he intends to use it, is 
sufficient in itself to make the difference between valid and in- 
valid results. : 

Sixth, and finally, the method makes claim to being scientiñc 
because the results can be checked and substantiated. The pro- 
cedure can be checked by comparing the results of different 
observers, or by repeating the study. Such checks, however, do 
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not insure validity; the validity of direct observation depends 
essentially upon the definitions of the acts which are to be re- 
garded as falling within the category being studied. Validity is 
the more difficult thing to check upon in most scientific tech- 
niques, and in many cases it must rest ultimately upon the 
consistency with which the findings of different scientific 
approaches point to the same general conclusions. The re- 
liability of observation can, however, be ascertained, and it has 
been found to be relatively satisfactory. 

Symonds gives a list of detailed requirements if the observa- 
tion is to be satisfactorily done. In his list of nine essentials 
to good observation he calls attention to the need for good eye- 
sight, alertness, the ability to estimate, the ability to dis- 
criminate, good physical condition, an immediate record, good 
perception, freedom from preconceptions, and emotional dis- 
interest.® Most of these constitute requirements which the 
observer himself must meet if he is to qualify as an expert, 
prepared to do scientific observation. 

Preparation for observing. One can secure a good idea of 
the steps he should take in preparing to observe by reading 
over a number of the studies that have been made since 1928 
and by noting the discussions that have been made of the 
difficulties which were encountered and the steps which have 
been recommended by the writers to correct these difficulties, 
The literature is not voluminous; one can read typical studies 
and treatises with a reasonable expenditure of time. Leading 
references have been given in the preceding section. 

The principal problems arising in connection with observing 
fall into four groups—planning the administrative aspects, de- 
fining the activities, arranging for the record, and training 
oneself to observe. Brief comments will be made on these 
phases of the work. 

Under administrative aspects are included those larger 
aspects of the study such as securing an appropriate group of 
persons to observe, deciding and arranging any special con- 
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ditions for the group, determining the length of each observa- 
tion period, the interval between periods, and the number of 
periods. These are in part problems of adjusting to local con- 
ditions, and not much can be said in general. On the other 
hand, one must consider the general principles of sampling, 
especially the facts of different numbers of children and dif- 
ferent numbers of periods. The length of the period of observa- 
tion must of course be related to the type of activity being 
studied, periods of a minute or so being suitable for some 
things and periods of an hour being better for others. Within 
limits of appropriateness, there is some evidence that a greater 
number of short periods is better than a smaller number of 
long periods.®° 

In defining the activities which are to be observed, the ob- 
ject should be, first, to include those activities which are true 
representatives of the general category which one is studying, 
and second, to define them so carefully that questions as to 
what is included do not arise.“ Obviously these two steps 
present real difficulty. They cannot be achieved without time, 
careful analysis of what one is trying to do, probably some 
reading, and actual observation in the field. The validity of the 
results will depend in large part upon how well one chooses 
and defines activities that will contain the essential elements in 
which he is really interested. Not infrequently the proper an- 
alysis can be made only after extended experience in the field 
—learning what to look for and deciding upon questionable 
acts. Perhaps it should also be said here that there is some 
danger in the tendency to restrict the trait being observed to 
as narrow or simple an element of behavior as possible; if 
carried too far, this often defeats the peculiar advantage of the 
observation technique in being able to deal with significant 
units of normal behavior. 


80 Ibid., pp. 32-38. 3 
81 Definition of what one is looking for is discussed at some length by Ralph W. 
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The definitions or specifications will, of course, not appear 
on the record form that is used; they must be well understood 
in advance. The form for recording will be arranged so as to 
make note-taking easy and rapid; a great deal of attention 
may well be given to this point. Much time can be saved by 
the appropriate use of symbols, abbreviations, and some use 
of shorthand. Suggestions can be secured from other studies 
of a similar nature, but in the end one will have to subject 
‘his record form to actual trial, along with critical analysis of 
the blank during and after trial. In designing a record form,” 
one will try to achieve convenience in analyzing or copying off 
the data from the original record. Although this is secondary to 
convenience in recording, it nevertheless should not be over- 
looked as a consideration. If the record is to be made by a 
stenographer, or moving-picture operator, or by some other 
special means, one should make preliminary or exploratory ob- 
servations in advance. 

The fourth phase of the work that should receive careful 
attention is training oneself or others to observe. Perception 
improves with practice. One learns to see characteristics of 
behavior by thoughtful practice in seeing them. Even a person 
who has become aware of the extent to which one has to learn 
to perceive certain characteristics of any situation will have 
little idea of the impossibility of his noting at the outset all 
of the instances of some act he is trying to catch. Every kind 
of research necessitates some expertness in technical skill, and 
observing is no exception. One is likely to think that such an 
everyday thing as “seeing” is commonplace and simple. This 
is not the case, because it is not the physical side of seeing that 
is the most important, but the psychological side, and this re- 
quires the building up of a recognition readiness before skilled 
seeing can take place. It may require several months of looking 
through a microscope before one can see things that are in 
plain sight to the trained person. One should expect to un- 


82 George A. Lundberg, Social Research, Chapter VI, “The Schedule as an Instrument 
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dergo training of several months in the observation of behavior 
characteristics before he begins to gather data for a study. 

Help in learning to observe may be secured from any system- 
atic, purposeful set of observations. A laboratory manual 
which is excellent for this purpose has been prepared by 
Wagoner.®* There are ninety-six exercises classified as to the 
type of characteristics to be observed. Although this book is 
designed primarily for nursery-school and kindergarten 
teachers in training, essentially the same procedures will be 
used by research workers. Further, one will find the kinds of 
things which are to be looked for in these different exercises 
suggestive of characteristics which he may desire to observe 
in a study of his own. A helpful bibliography is included. 

In closing, the worker should be cautioned against under- 
estimating the preparation which is essential to success in the 
use of this technique. Stoddard comments,"* “The observa- 
tional method used to be popular because it required no prepa- 
ration and no apparatus. One simply watched the child, and 
the amount he was able to select and observe accurately was 
dictated by his conscience or his genius.” But that day has 
passed, and the work done under such concepts is one of the 
things which educational research has to live down. 


6. STUDIES EMPLOYING SURVEY-APPRAISAL PROCEDURES 


A. Procedures which involve direct judgment. Appraisal 
procedures are divided into two groups for convenience in 
treatment, the first group involving direct judgments, and the 
second group being more objective in immediate form, but 
involving judgment in the preparation of the instruments or 
the selection and treatment of the factors which are repre- 
sented, The first group produces a type of data which are 
different from most of those that have been discussed under 
other types of normative-survey research. Tests and question- 
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naires which call for the evaluation of certain items, particu- 
larly when a choice from among several scaled opportunities 
is called for, yield the same type of data as the procedures 
treated in the present section. 

Appraisal is a form of classification or scaling according 
to subjective values. If an instrument is used which makes 
the procedure relatively objective, the instrument must have 
been calibrated in terms of human judgment when it was 
constructed. In the procedures which are more directly sub- 
jective, the value which is assigned to any specimen being 
rated depends more or less on the individual doing the rating, 
and it may vary with the same individual from one time to 
another, Direct rating is perhaps the least satisfactory of the 
data-gathering techniques from the scientific point of view. 
On the other hand, there are many practical problems for 
which it is the most important of all research techniques. Ap- 
parently, the more nearly we get to the heart of human prob- 
lems, the less completely we can depend on approaches that 
are acceptable to those who emphasize the formal character- 
istics of objective science. In these areas one has to choose 
between toyalty to formalism and the desire to do something 
of practical worth, even though it is not entirely above criti- 
cism. 

For those who feel that all is lost whenever a research 
worker allows judgment to enter into his data, it should be 
pointed out that social scientists are confronted with problems 
which differ fundamentally from those of the physical scien- 
tist, and they must utilize procedures which are appropriate 
to their problems. The criterion of the social sciences cannot 
be how nearly they approach the concepts which have proved 
useful in the physical sciences, but rather how well they deal 
with the problems which comprise their own responsibility. 
Because the social sciences are in their infancy, early attempts 
in the more complicated areas of their field will naturally be 
primitive; ìt is their task, however, not to say that they will 
avoid certain things because these can only be done imper- 
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fectly, but rather to use crude beginnings as a starting point 
for a long road of discovery and constant improvement, both 
in techniques and in findings. In making this progress, these 
sciences must be oriented by the search for truth that is 
important in their field and must use increasingly satisfactory 
measures that will yield this truth, whatever these measures be. 

It should not be thought, however, that even objective sci- 
ence of the usual sort is entirely free from observational error. 
All measurement—even measuring short distances—contains 
some inexactness and, if we make the units of measurement 
small enough, the observation will vary for the same person 
from time to time and from one observer to another. Astrono- 
mers long ago noted this fact and denoted such differences in 
observation as the personal equation, or observational errors. 
There is a definite statistical theory for dealing with such 
errors in the physical sciences, known as the theory of least 
squares, and this has been used for many years. The more 
modern theories of physics, introduced since the work of Bohr 
in rgr2 and modified further by discoveries of the last five 
years, recognize certain variations in the behavior of matter, 
and thus add irregularities in the object to those which may 
occur in the observation. No science—not even abstract 
mathematics—can claim to be entirely free from error or 
approximation. 

Appraisal leans more heavily upon the human element than 
do the objective sciences, for appraisal is undertaken for the 
specific purpose of including the human element. It is not an 
attempt to measure objective characteristics; that can be done 
by the usual methods of the physical sciences. It is rather an 
attempt to determine the effect of these characteristics upon 
human beings. It is concerned primarily with human values 
and secondarily with the physical attributes to which these 
values are, somewhat ephemerally, attached. Our efforts to 
improve the techniques of appraisal will not, therefore, be 
directed toward eliminating those changes in value which are 
truly characteristic of human preferences, for to do so would 
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be to inject spurious elements into what we desire to keep 
natural. We should strive only to free the expression of value 
from sporadic elements so that we may secure a judgment, of 
the essential characteristics of specimens that is true or typical 
for the individual judge at a given time. We must expect 
reasonable variation from time to time, or place to place, or 
between one judge and another. 

At present the foregoing position is not widely accepted 
among research workers, who are likely to attempt to apply to 
evaluative procedures the same criteria that they would to a 
yardstick, Human beings, however, are something more than 
inert measuring sticks; they have a large number of very 
unstable characteristics of response, and the canons of science 
cannot invalidate their right as human beings to have wants, 
express opinions, and make choices, however changeable these 
may be. Appraisal is simply the procedure by which we secure 
and make overt these characteristically variable reactions. 
Whatever its scientific standing, we recognize that for certain 
purposes an index of human values may be more important 
than any number of physical measurements made perhaps with 
incredible accuracy and reliability. 

The discussion will turn at this point to types of rating and 
various applications that have been made of them. These will 
be introduced in the order of their increasing complexity or 
refinement of technique. As was noted in connection with 
survey testing, these procedures enter into research in various 
ways. They are treated here, not as processes which normally 
represent complete research studies in themselves, but as 
techniques which enter into research undertakings in various 
ways and to various degrees. Their relation to a complete re- 
search investigation depends upon the nature of the par- 
ticular study. In some instances the techniques are themselves 
the object of investigation. 

Direct comparison. Apart from simple estimation of the 
worth of any object by itself, the most direct form of evalua- 
tion is the comparison of various specimens that can be brought 
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vogether, studied, and ranked in order. This form of appraisal 
is most likely to occur in the construction of a schedule or 
scale which is later to be used in rating. It is, however, some- 
times used directly. An investigator secures the coöperation 
of a number of persons, referred to as “judges,” who rank or 
rate in some other way the objects with which he is concerned. 
This process is commonly referred to as the “jury technique,” 
since it involves the pooled judgments of a number of persons 
—sometimes several hundred or more. 

Perhaps the commonest form of this technique is the rating 
of items in a list. We may refer again to the study of items 
for school records, made by Heck.*® Four series of frequencies 
were secured, and these data were submitted with the items 
to a group of 133 judges who were requested to pick out the 
items of basic and universal value. On the basis of the judg- 
ments of this “jury,” the final list was compiled. In an attempt 
to ascertain the most pressing problems in education, Hurd ** 
scanned current educational literature and secured a list of 
102 constructive criticisms. He submitted these to 154 judges 
to pick out the most important ones. Forty-three of the items 
were rated as very important by at least 80 per cent of the 
judges, and these forty-three were reported by Hurd as im- 
portant points for foci of attention in making educational 
progress. 

An illustration of the jury technique on a large scale is to 
be found in the Commonwealth Teacher-Training Study.*? 
One part of this study had for its purpose the ascertaining of 
teacher traits that are important for successful work. A master 
list of twenty-five traits was secured by interviews and subse- 
quent analysis. This phase of the study has been discussed 
earlier. This master list was submitted to twenty-five judges 
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(experienced school administrators) to rate the traits on the 
basis of importance. Separate ratings were made for each of 
five groups of teachers: senior high school, junior high school, 
intermediate, primary, and rural schools. The judges were 
asked to designate each trait as most important, of average 
importance, or of least importance. The frequency with which 
each trait was placed in one of these three categories was 
then used to derive finer gradations, so that complete ranking 
of the twenty-five traits was finally deduced.’ 

As a second part of this same study, a master list of teacher 
activities (duties) was rated on four aspects. The frequency 
of performance of the duties was determined by submitting 
the list to groups of teachers. For ratings on the other three 
aspects, the check-list was submitted to groups of judges— 
usually twenty-five to a group, since this number was found 
to yield a reliable judgment. These groups included adminis- 
trators, critic teachers, graduate students, etc., and were so 
selected as to represent many different backgrounds. They 
were asked to rate each item on a scale of three degrees, for 
each of the three aspects—difficulty of learning to perform the 
activity, the importance of the activity, and the desirability 
of learning the activity in a teacher-training school rather 
than learning it on the job or in life at large. Each of the 
four different aspects was found to contribute something 
unique to the picture of the activity.®° 

The technical details of evaluating the ratings and of com- 
bining them into a composite score are discussed in the study.” 
The results of the four ratings and the resulting standing of 
the activities on the composite scale are given for each of five 
groups of teachers.” These are shown separately for different 
groups of judges. This presentation represents probably the 
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most elaborate and extensive piece of work of its kind in edu- 
cational literature. 

The ranking of a group of human beings may be regarded 
as constituting a second type of direct comparison. One may 
be requested to rank the pupils in a class, or to rank the indi- 
viduals of any other group on one or more characteristics. 
Sometimes ranking takes the place of other marks or ratings 
given pupils on their classroom work.” Ranking does not 
necessarily involve the assignment of a particular numerical 
value to each individual; it only requires judging that one 
individual is higher or lower than another one in the group, 
with respect to the particular character on which they are 
being ranked. There are three procedures for ranking, known 
respectively as the method of paired comparisons, the method 
of comparative arrangement, and ranking from a random list. 
These procedures are described in appropriate treatises.” The 
method used by Charters and Waples represents a ranking 
derived from a numerical composite of ratings. 

A third type of direct comparison is afforded by scaled speci- 
mens, such as a handwriting scale or a composition scale. The 
user compares the specimens to be rated with those which con- 
stitute the scale to determine whether his specimen is better 
or poorer than a certain sample in the scale. When he finds 
the approximate position which his specimen would occupy 
on the scale, he assigns the appropriate scale value to it. This 
requires less judgment than rating without the scale. 

This is a common procedure where readily handled products 
are to be rated. Thorndike introduced this procedure early in 
the measurement era with his handwriting scale,** which was 

92A. Duryee Crooks, “Marks and Marking Systems: A Digest,” Journal of Educa- 


tional Research, XXVII (December, 1933), 259-72. See especially, “Ranking,” pp. 
263-4. 

93 Florence L. Goodenough and John E. Anderson, Experimental Child Study, pp. 
413-8, New York: The Century Co., 1931. Pp. 546. See also “Ranking Methods,” 
in the Index of Subjects. 

Percival M. Symonds, Diagnosing Personality and Conduct, pp. 54-5, 76-8. New 
York: The Century Co., 1931. Pp. 602. See also “Ranking,” in the Index. “ 

94 Edward L. Thorndike, “Thorndike’s Scale for Measuring the Handwriting of 
Children.” New York: Bureau of Publications, Teachers College, Columbia University, 


1910; revised, 1912. 
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followed shortly afterwards by Ayres’ scale,” by Thorndike’s 
drawing scale,°* and by Freeman’s diagnostic handwriting 
scale.” Direct comparison has been a favorite method for the 
measurement of composition; Hillegas introduced his compo- 
sition scale, consisting of ro sample compositions, in 1912.°* 
This scale has been followed by a number of others, being 
climaxed perhaps by the extensive scales of Odell for rating 
answers to thought questions.” Other products have been 
rated in similar fashion.t 

The use of such scaled specimens is an attempt to make 
individual judgment representative of average or group opin- 
ion. In some cases, it enables the individual to know the opin- 
ion of experts. The scaled specimens also afford an objective 
representation of values, which aid in preventing undue varia- 
tion in judgment from time to time. In this way idiosyncrasies 
and wanton caprices of individual judgment tend to be re- 
duced, and a more typical, more true evaluation by the indi- 
vidual is secured. 

1. Scaling specimens. Perhaps this is the most appropriate 
time to note the procedure for scaling a set of specimens to 
serve as standards of judgment. The procedure of scaling by 
group judgment is in reality an example of the first form of 
comparison, directed, however, not toward the purpose of im- 
mediate end products, but toward the production of instru- 
ments with which to do further work. The technique has been 
well described in a number of places. Thorndike made an 


95L. P. Ayres, “Ayres’ Handwriting Scale: Three-Slant Edition,” rọ12. (Revised in 
1917 and published as the Gettysburg Edition.) New York: Russell Sage Foundation. 

96 “Thorndike Drawing Scale,” 1913. Revised and extended in 1924, as the “Thorndike 
Scale for the Merit of Drawings by Pupils.” New York: Teachers College, Columbia 
University. 

97 Frank N. Freeman, “Freeman Chart for Diagnosing Faults in Handwriting.” Boston: 
Houghton-Mifflin Co., 1914. 

98 Milo B. Hillegas, “Hillegas Scale for Measurement of English Composition by 
Young People.” New York: Bureau of Publications, Teachers College, Columbia Univer- 
sity, 1912. 

99C. W. Odell, The Use of Scales for Rating Pupils’ Answers to Thought Questions. 
Bureau of Educational Research Bulletin, No. 46. Urbana, Ill.: University of Illinois, 
May, 1929. Pp. 34. The scales in question are published at the same place. 

100 Rating scales consisting of scaled specimens will be found in general references on 
achievement testing. For these, see footnote 11 of the preceding chapter. 
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early contribution in preparing his handwriting scale.1° Ayres 
employed a different basis, the length of time required to read 
specimens, as the criterion of legibility of handwriting.1° 

The field of composition presents a notable series of studies 
in scale construction, beginning with the work of Hillegas + 
and including successive contributions by Thorndike, Willing, 
Trabue,!°** VanWagenen, Hudelson, S. A. Leonard, ° and 
others, Odell’s work on scoring essay tests *°? should also be 
consulted. One writer, in fact, suggests that these are excellent 
composition scales. This whole field of scale construction for 
composition has been reviewed in detail by Lyman,” who 
includes also a discussion of criticisms of the scales, as well as 
studies of their use. 

The technique of constructing attitude scales utilizes the 
same fundamental principle, with special adaptations. These 
are presented by Thurstone and Chave *™% and include impor- 
tant contributions to theory by Thurstone.“° Reviews and 


101 Edward L. Thorndike, “Handwriting,” Teachers College Record, XI (March, 1910), 
83-175. 

teat, P. Ayres, “A Scale for Measuring the Quality of Handwriting of School 
Children,” Bulletin E-113. New York: Russell Sage Foundation, 1912. Pp, 16. 

108 Milo B. Hillegas, “Scale for the Measurement of Quality in English Composition 
by Young People,” Teachers College Record, XIII (September, 1912), 331-84. Also 
issued as a separate publication: Teachers College, Columbia University, 1912. Pp. 54. 

104 Marion R. Trabue, ‘Supplementing the Hillegas Scale,” Teachers College Record, 
XVIII (January, 1917), 51-84. Also published separately: Teachers College, Columbia 
University, 1922. Pp. 38. 

105 Earl Hudelson, English Composition: Its Aims, Methods, and Measurement, 
Twenty-Second Yearbook of the National Society for the Study of Education, Part I, 
Pp. r-r00, Bloomington, Ill.: Public School Publishing Co., 1923. Pp. 173. 

106 Sterling A. Leonard, “Building a Scale of Purely Composition Ability,” English 
Journal, XIV (December, 1925), 760-75. 

107 Odell, op. cit., (footnote 99). 

108R, L. Lyman, Summary of Investigations Relating to Grammar, Language and 
Composition, Chapter IV, “Investigations in the Field of Written Composition,” pp. 
134-73. Supplementary Educational Monographs, No. 36. Chicago: University of Chicago, 
January, 1929. Pp. 302. ; 

109L. L. Thurstone and E. J. Chave, The Measurement of Attitude, Chicago; Uni- 
versity of Chicago Press, 1929. Pp. 97. . 

110 L. L. Thurstone, “A Mental Unit of Measurement,” Psychological Review, XXXIV 


(November, 1927), 424-32- i 4 
, “An Absolute Zero in Intelligence Measurement,” Psychological Review, XXXV 


(May, 1928), 175-97- 7 
Z ie ee Often Noticed Differences,” Journal of Educational Psychology, 


XVIII (May, 1927), 289-93. 
y S peyckophysial ‘Analysis,’ American Journal of Psychology, XXXVIII (July, 


1927), 368-89. 3 3 
B The Law of Comparative Judgment,” Psychological Review, XXXIV (July, 


1927), 273-86. 
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analyses are cited in footnotes 51, 52, and 54 of Chapter VII 
and will not be repeated here. One may be interested in reading 
early statements of Thorndike ™* and Ayres, made at the 
famous First and Second Annual Conferences on Educational 
Measurements at Indiana University. The underlying theory 
of scaling runs well back into the nineteenth century. For a 
_ key to this literature the reader may consult Kelley ™ and 
also the discussion in Brown and Thomson.''* Very simple 
statements are given in Holzinger “5 and in Garrett '*° for 
those who want a one-page presentation. Kelley ™" gives a 
somewhat more detailed mathematical statement. 

The procedure in establishing weights for items in a score 
card is similar in nature to the scaling of specimens, though 
the theory has not been well explored. The method will be 
found in later references on score-card construction. 

Check-lists. Schedules are sometimes prepared which consist 
essentially of a list of items with a place to check, or to mark 
“yes” or “no.” The chief purpose of the check-list is to call 
attention to various aspects of an object or situation, to see 
that nothing of importance is overlooked. A check-list may 
be used to direct attention to various large aspects, or to check 
against the completeness of details, according to the nature 

111 Edward L. Thorndike, “Tests of Oral and Silent Reading,” pp. 37-51, in Pro- 
ceedings of a Conference on Educational Measurements, Bulletin of the Extension 
Division, Indiana University, XII (September, 1914), No. 10. Bloomington, Ind. Pp, 170. 

112L. P. Ayres, “The Measurement of Educational Processes and Products,” pp- 
127-35, and ‘Making Education Definite,” pp. 85-96, in Second Annual Conference on 
Educational Measurements. Indiana University Bulletin, XIII (October, 1915), No. 11. 
Bloomington, Ind. Pp. 221. 

118 Truman Lee Kelley, Interpretation of Educational Measurements, pi 13. Yonkers- 
on-Hudson, N. Y.: World Book Co., 1927. “Standardized Judgments,” and the references 
given (somewhat fragmentarily) for these names in the Bibliography, pp. 349-53- 

114 William Brown and Godirey H. Thomson, The Essentials of Mental Measurement, 
Chapters I and 11. (Revised edition.) Cambridge: Cambridge University Press, 1921. 

ip. 216. 

115 Karl J. Holzinger, Statistical Methods for Students in Education, p. 228. Boston: 
Ginn and Co., 1928. Pp. 372. 

116 Henry E. Garrett, Statistics in Psychology and Education, pp. 108-9. New York: 
Longmans, Green and Co., 1926. Pp. 317. 

Some of the classical background is portrayed by this same author's Great Experiments 
in Psychology, Chapter XII, “Weber's and Fechner’s Laws, and the Rise of Psycho- 
physics,” pp. 266-93. New York: The Century Co., 1930. Pp. 337. He gives four 
references which are in English. 


117 Truman L. Kelley, Statistical Method, Sec. 92, “Psychophysical Methods,” pp. 
326-30. New York: The Macmillan Co., 1923. Pp. 390, 
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of the instrument used. It may afford simply a number of 
suggestions of what is usually desirable to do. The check-list 
may be useful as an aid to recording, such as in observing be- 
havior; this use was discussed in an earlier section of this 
chapter. 

The check-list may also be used as a sort of scale, yielding 
a score. It is with such use that we are chiefly concerned in 
this section. Noting whether a characteristic is present or 
absent, and counting the number of favorable or unfavorable 
items that are checked afford a total that may be compared 
with other similar totals to indicate the general satisfactoriness 
of the thing evaluated. This process corresponds to a true- 
false test, in which the pupil is scored dichotomously, either 
as knowing or not knowing each item, the total score being 
indicative of the breadth of the pupil’s acquaintance with the 
subject-matter, 

Determining the presence or absence of a given character- 
istic may require little or no judgment, or it may require the 
exercise of considerable judgment. Check-lists which relate to 
highly objective aspects of a situation belong properly with 
the second set of appraisal procedures (to be discussed later), 
but no attempt will be made to divide check-lists into two 
groups for the purpose of separate treatment. The use of 
check-lists in observing the larger outcomes of education and 
in measuring personality has already been discussed. The gen- 
eral discussion of check-lists will be given at this place. 

A number of check-lists have been prepared for the ap- 
praisal of school buildings, school property, and school-building 
plans. For example, one check-list**® contains the points 
one should look for when checking over plans, details, and 
specifications of the architect, including such aspects as heat- 


118 For references containing examples, see the topic, “Score Cards and Building 
Measurement,” in Bibliography of Schoo? Buildings, Grounds, and Equipment, by Henry 
Lester Smith and Forest Ruby Noffsinger. This bibliography is published in four parts, 
in different numbers of the Bulletin of the School of Education, Indiana University. 
See Part II, Vol, IX, No. 2 (March, 1933), 40-2; and Part IV, Vol. XI, No. 2 (March, 


1935), 32 
119 «Check: Lists for School Buildings,” Architectural sree LV (December, 1931), 


772-4. 
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ing, ventilation, plumbing, design, materials, electrical work, 
and other special features. The reader is referred to the litera- 
ture cited for a fuller acquaintance with such check-lists. 

As illustrative of check-lists for the business phases of school 
administration the following may be mentioned: Linn’s check- 
list for possible economies,'*° and various check-lists and 
specifications for the purchase of school supplies.t?? General] 
administration, including provision for instruction, is repre- 
sented by the following check-lists: Z. R. Clark’s list for ap- 
praising the superintendent’s report,'** the Frederick-Schorling 
check-list for the organization of junior high schools,’** and 
another for the organization of senior high schools,'** and 
Wade’s general check-list for all aspects of the secondary 
school, from the pupils standpoint.’** The Committee on the 
Revision of Standards for Higher Institutions, of the North 
Central Association, used a check-list technique to secure and 
evaluate certain college procedures which could be counted 
only as present or absent. Presumably check-lists will con- 
stitute a part of their final procedure also. The Research 
Division of the National Education Association has prepared 
check-lists, with standards, which may be used in studying 
the provisions and practices of state school systems.'*? The 

120H. H. Linn, “Check-Lists for Determining Possible Economies in Public-School 
Administration,” American School Board Journal, LXXXVII (July-November, 1933). 
See also his Practical School Economies. New York: Teachers College, Columbia Uni- 
versity, 1934. Pp. 461. 

121 See the references given on p. 363, of “The School Plant,” Review of Educational 
Research, V (October, 1935), No. 4. 

122 Zenas Read Clark, The Recommendation of Merit in Superintendents’ Reports to 
the Public. Teachers College Contributions to Education, No. 471. New York: Teachers 
College, Columbia University, 1931, Pp. 123. 

128 “Frederick-Schorling Standardized Check-Lists for the Organization of Senior High- 
School Grades, and Standardized Check-Lists for the Organization of Junior High-School 
Grades.” Ann Arbor, Mich.: Ann Arbor Press, 1933. 16 pages each, 

124 J. Thomas Wade, A Measurement of Secondary School as a Part of the Pupil's 
Environment. Teachers College Contributions to Education, No. 647. New York: 
Teachers College, Columbia University, 1935. Pp. 68. 

125 John Dale Russell, “Measuring the Excellence of Institutions of Higher Education,” 
pp. 83-94, in Twenty-Second Annual Conference on Educational Measurements. Bulletin 
of the School of Education, Indiana University, XII (December, 1935), No. 1. 
Bloomington, Ind.: Bureau of Coiperative Research. Pp. 97. See especially p. 90. See 
also later footnotes, numbers 287, 292, and 293. 

126 “A Self-Survey Plan for State School Systems,” Research Bulletin of the National 


Education Association, Part I, “Check-Lists,” VIII (March, 1930), 54-87; and Part II, 
“Handbook,” VIIT (May, 1930), 90-163. 
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Research Division points out that more will be gained from 
using the check-lists as a starting point for study than merely 
for immediate evaluation. This is a wholesome suggestion 
applicable to all instances of rating. 

Check-lists are also applied to classroom instructional activi- 
ties. References by Wrightstone and Pistor have already been 
given.'*? This is a developing field. Franzén has analyzed both 
general and specific aims in the teaching of algebra into de- 
tailed pupil and teacher activities; his list +° may be used as 
a check-list by teachers or possibly supervisors. Tyler has 
prepared a check-list to be employed by an observer in study- 
ing the working habits of students in using a microscope.12° 
A student is directed to find a certain specimen in a solution, 
While he is doing this, the instructor notes the actions which 
he makes, the length of time taken, and finally, success of 
the student, and the quality of his mount. For this purpose the 
instructor uses a check-list which contains all of the likely 
actions, both desirable and undesirable. The instructor writes 
numbers on the check-list to indicate the sequence of the stu- 
dent’s actions. This procedure is both diagnostic and evalu- 
ative. 

The use of check-lists in the supervision of classroom in- 
struction was one of the early applications of this technique. 
This field is somewhat distinct from that of observing out- 
comes of instruction, being concerned more with the activities 
of the teacher, though it is generally conceived of as embrac- 
ing to some degree the quality of the educational product. 
Burton is credited with being the first to make use of check- 


127 See footnotes 61 and 63. See also: J. Wayne Wrightstone, “Instrument for 
Measuring Group Discussion and Planning,” Journal of Educational Research, XXVII 
(May, 1934), 641-50. See other articles listed for Wrightstone in the Education Index. 

128 Carl G. F. Franzén, “Measuring the Efficiency of Teaching First Year Algebra—A 
Technique in Supervision,” Twenty-Second Annual Conference. on Educational Measure- 
ment. Bulletin of the School of Education, Indiana University, XII (December, 1935), 
pp. 32-50. Bloomington, Ind.: Bureau of Coöperative Research. Pp. 97. See especially 
TAN W. Tyler, “A Test of Skill in Using a Microscope,” Educational Research 
Bulletin; IX (November 19, 1930), 493-6. Also reprinted in his Constructing Achieve- 
ment Tests, pp. 37-41. Columbus, Ohio: Bureau of Educational Research, Ohio State 


University, 1934. Pp. 102. 
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lists in connection with supervisory studies.**° Since that time 
check-lists for the observation of classroom activities and 
teacher-pupil relationships have gone through various stages 
of development, and many studies of their use have been 
made.'° The appraisal of the teacher’s instructional activities 
not infrequently depends upon the observation and rating of 
characteristics in addition to those which might be gathered 
on a check-list; some of these procedures were treated in the 
section on “Observation of Behavior,” and others will be given 
in the next section, on “Rating Scales.” 

The check-lists which were built up in the Commonwealth 
Teacher-Training Study may be used in a number of ways, 
hoth for analysis and for evaluation, The extensive list of 
teacher activities,“ particularly with the frequency and rat- 
ings attached to each activity, provides an important means 
of checking courses of study for the training of teachers. An- 
other use of the list is the checking of textbooks which deal 
with topics related to teaching. Charters and Waples state, 
for example: 


A discussion of school records and reports by a well-known author- 
ity specifically treats of but twenty-five of the 153 duties contained 
in the master list that relate to recording and reporting, and the ma- 
jority of the activities treated are below median importance. (p. 30.) 


The authors point out that one may go farther in analyzing 
textbooks by noting the depth of treatment of each topic. 
They suggest using five degrees: (1) the idea is merely men- 
tioned; (2) the idea is discussed in sufficient detail to make 
its meaning clear to some one who is unfamiliar with it; (3) 
the difficulties commonly met in making use of the idea are 
stated; (4) possible methods of overcoming the difficulties are 

130A, S. Barr, Introduction to the Scientific Study of Classroom Supervision, pp. 5; 
and 198 ff. See especially the references in the footnotes, and those at the end of Chap- 
ter V. New York: D. Appleton and Co., 1931. Pp. 399. 

See the Education Index, topic: “Supervision and Supervisors,” for an occasional article 
dealing with the use of check-lists. 

181 W. W. Charters and Douglas Waples, The Commonwealth Teacher-Training Study, 


PP. 493-535. The lists appear in complete form, but without evaluation, in pp. 257-303, 
or pp. 304-472. Chicago: University of Chicago Press, 1929. Pp. 666. 
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listed; and (5) a statement of the principles, or sources of 
authoritative support, for the methods given, are included.” 
Other definitions of degrees may be more appropriate for’ the 
content of a given book. However, the procedure offers a 
method of indexing the fullness of treatment accorded topics 
of varying degrees of importance to teachers. Johnson has 
prepared a check-list which applies to other aspects of text- 
books,1** 

Personality is sometimes rated by means of a check-list. 
It would be possible, for example, to use the list of traits and 
trait-actions derived in the Commonwealth Teacher-Training 
Study *** as a check-list for the personality traits of a school 
worker. Hartshorne, May, and Maller made use of check-lists 
in the Character Education Inquiry. 

Bixler has prepared a check-list for research studies +° 
which investigators may use to check over their research plans. 
The list applies to practically every kind of quantitative re- 
search, and students will find it useful, whatever their prob- 
lem. It not only suggests steps which one might overlook but 
indicates the manner in which it is desirable to do things. 

Several of the studies which utilize check-lists contain de- 
scriptions of the procedure by which they were prepared. One 
who is considering constructing a check-list will do well to 
examine closely those that have been made, and to read as 
many descriptions as possible, particularly the references by 
Charters and Waples, Frederick,** Wade, Tyler, and Bixler; 
also the references given earlier in connection with “Observa- 


132 Ibid., “Checking the Adequacy of Textbook Content,” pp. 137-42. 

133 Franklin W. Johnson, “Checking Lists for the Selection of High School Text- 
tooks,” Teachers College Record, XXVII (October, 1925), 104-8. 

134 W. W. Charters and Douglas Waples, op. cit., Part II, Section I, “A List of 
‘Teachers’ Traits and Trait-Actions,” pp. 223-44. Contains 83 traits with about ro actions 
under each one. 3 

135 Hugh Hartshorne, Mark A, May, and Julius B. Maller, Studies in Service and 
Self-Control, pp. 91-3. Studies in the Nature of Character, II. New York: The Macmillan 
Co., 1929. Pp. 559. Also described in Symonds, (footnote 149), pp. 72-3. 

136 H. H. Bixler, Check Lists for Educational Research. New York: Bureau of Pub- 
lications, Teachers College, Columbia University, 1928. Pp. 118. 

137 Orie I. Frederick, “Two Standardized Check-Lists for the Organization of Secondary 
Schools.” Ann Arbor, Mich.: The Author, January, 1933. Pp. 64. See Chapters II, 
III, V, and VI. 
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tion of Behavior” (footnotes 61-63) and most of the volumes 
in the series of Child Development Monographs (footnotes 
56 and 57). Any good treatise on the use of a check-list will 
throw some light on the construction of such an instrument. 

Rating scales. The most commonly used instrument for mak- 
ing appraisals is probably the rating scale. There are a large 
variety of forms and uses of rating scales. Typically, they di- 
rect attention to different parts or aspects of the thing to be 
rated, but not as many as the check-list or score card—nor- 
mally about three to ten aspects. They provide a scale for 
assigning values to each of these aspects. The scale may take 
any of a number of different forms; it may be simply a series 
of numbers, or a graduated line, or qualitative terms such as 
“Good,” “Poor,” etc., or a series of named attributes peculiar 
to each scale, or a series of carefully worded descriptions of 
states representing different degrees of each aspect to be rated. 
Sometimes a special scale is eliminated, and the objects are 
rated on each characteristic in terms of per cent. Some of 
the applications of rating scales will be discussed. 

Possibly we should regard carefully codified criteria, or 
standards, as constituting a sort of inchoate rating scale. The 
essence is there, but not the form. A number of these exist 
and serve to orient judgment. Examples of criteria for the 
appraisal of the school plant,’** playgrounds,” and written 
English ™° will be found in the references cited. In addition 
there are sets of standards which accompany building score 
cards,'** which one might use directly, if he desired. 


188 See for example, Arthur B. Moehlman, “Appraising the Existing School Plant,” 
Nation’s Schools, VII (March, 1931), 84, 86, 88, 90; also, “Location and Use as Factors 
in Plant Appraisal,” Nation's Schools, VIL (June, 1931), 78, 80, 82. 

139 National Recreation Association, Report of Committee on Standards in Playground 
es 1931. Also, Space Requirements jor the Children's Playground, 1934. New 

OFS. 

140Z. i. Huxtable, “Criteria for Judging Thought Content in Written English,” 
Journal of Educational Research, XIX (March, 1929), 188-95. A separate rating scale 
also is publishea. 

141 George D. Strayer and N. L. Engelhardt, Standards for Elementary School Buildings, 
Revised and enlarged, 1933. Pp. 181. Standards for High School Buildings, 1924. Pp. 95- 
With W. S. Elsbree, Standards for the Administration Building of a School System, 1927- 
Pp. 40. Standards for Village and Rural School Buildings of Four Teachers or Less, 
1920. Pp, 22. N. L. Engelhardt, Standards for Junior High School Buildings, 1932- 


THE NORMATIVE-SURVEY METHOD 425 


A simple form of rating scale is commonly employed in 
judging contests of various kinds, such as speaking and music 
competitions. One may be asked to judge a debate, giving 
perhaps 70 per cent of the weight to content and 30 per cent 
to form of presentation. These directions constitute a rudi- 
mentary form of rating scale. They mention two aspects and 
assign the weight to be given to each, the scale being per cent. 
For the judging of choral music performance, one good rating 
scheme divides the consideration into six aspects—accuracy, 
tone, diction, rhythm, phrasing, and interpretation and gen- 
eral effect. Judges may be asked to give equal weight to 
each of these or to give some other specified weighting. 
Usually such divisions and weights are based on no formal 
study, though considerable thought may be devoted to them. 
Such a division as that referred to for music represents the 
consensus of opinion of experts who have analyzed the field 
and given critical thought to trying out the scheme set up. 

1. Teacher rating. Rating scales have been put to extensive 
use in the field of rating teaching and teachers.’** This proc- 
ess, of course, extends to the selection of teachers, and the 
prediction of teaching success. A considerable number of such 
rating scales have been prepared and subjected to much sta- 
tistical study.*** Although common sense and experience seem 
Pp. 161. The above are all published by Teachers College, Columbia University, New 
“Tc. Holy and William E. Arnold, Standards for Junior High School Buildings. 


Bureau of Educational Research Monographs, No. 17. Columbus: Ohio State University, 
1932. Pp. 62. Also in the American School Board Journal, LXXXVI (January, March, 
and April, 1933). j 

142 Sir Hugh S. Robertson, Choir Singing. Festival Booklets, No. Two. London: 
Paterson Sons and Co., Ltd. Pp. 46. n.d. hd n 

143A good general discussion of rating teachers is given in Charles W. Knudsen, 
Evaluation and Improvement of Teaching, Chapter VI, “The Use of Rating Devices in 
the Evaluation of Teaching,” pp. 205-48. Garden City, N. Y.: Doubleday Doran and Co., 
Inc., 1932. Pp. 538. Includes a bibliography of eighty-two references to articles and 
rating scales, re, tat ; 

A. S. Barr and Lester M. Emans, “What Qualities Are Prerequisite to Success in 
Teaching?” Nation’s Schools, VI (September, 1930), 60-4. Reported in summary form 
in A. S. Barr, An Introduction to the Scientific Study of Classroom Supervision, pp. 
328-40. New York: D, Appleton and Co., 1931. Pp. 399. é 

144 See the discussions, and the references, in the following: T. I. Torgerson, ‘The 
Measurement and Prediction of Teaching Ability,” Chapter II, pp. 261-6; also, “Technics 
in Selection and Placement,” p. 282, in “Teacher Personnel,” Review of Educational 


Research, IV (June, 1934), No. 3- % 
Russell L. = Butsch, “Teacher Rating,” Chapter VII, pp. 99-107. in “Teacher 
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to indicate that rating scales must be a help in appraisal, 
statistical analysis gives disappointing results." Barr finds 
evidence, however, that their use is superior to judgment of 
teaching merit when unaided by any kind of analytical scale. 
Torgerson points out, probably with considerable justification, 
that the low correlations between the results of rating scales 
and other criteria may be the fault of the criteria." The es- 
tablishing of valid criteria is one of the most difficult tasks 
in the validating of research instruments, and it is at the same 
time one of the most essential steps. 

In reviewing these techniques, Anderson calls attention to 
another important factor, namely, that “specific traits and 
characteristics may be helpful or detrimental to a teacher ac- 
cording to the combinations in which they occur in him.” '* 
The pattern or organization of personality characteristics is a 
matter that has not received adequate attention in research. 

A different rating procedure is provided by the Probst Serv- 
ice Ratings.” The report forms do not consist of a series 
of scales, but simply of statements which describe various 
forms of the individual’s behavior, such as, “Talks too much,” 
“Accurate but very deliberate,” “Works well without super- 
vision.” These appear in mixed order. Some statements are 
obviously favorable and some obviously unfavorable; others 
are indeterminate (on the surface). The rater checks those 
statements which he believes to be descriptive of the perform- 
ance of the individual being rated. The check marks are then 
scored by a stencil with weights which have been statistically 


Personnel,” Review of Educational Research, I (April, 1931), No. 2. See also p. 244 of 
sic of Personality and Character,” Review of Educational Research, II (June, 1932), 

o. 3. 

For current references see “Teacher-Rating,” in the Education Index and in the annual 
Bibliography of Research Studies in Education of the United States Office of Education. 

145 A. S. Barr, “The Measurement of Teaching Ability,” Journal of Educational 
Research, XXVIII (April, 1935), 561-9. 

146 Earl W. Anderson, “Techniques of Research Used in the Field of Teacher Per- 
sonnel,” p. 16, in “Methods and Techniques of Educational Research,” Review of Edu- 
cational Research, IV (February, 1934), No. 1. 

147 J. B. Probst, Service Ratings. Technical Bulletin No. 4. Published as one of a 
series of joint publications by the Bureau of Public Personnel Administration and the 
Civil Service Assembly of the United States and Canada. Chicago: 923 East Sixtieth St., 
1931. Pp. 94. 
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derived. The procedure avoids an awareness of how high or 
low one is rating another; it carries definite limitations, how- 
ever. It was originated for rating employees in public service; 
a special form has been prepared for rating teachers. The 
items are weighted heavily on the administrative aspects of 
a teacher’s work, and inadequate attention is given to in- 
structional processes. The device is suggestive for further re- 
search, 

2. Rating personality. Closely related to this field of teacher 
rating is the general area of personality and character rating. 
This differs from measurement of the same qualities by tests, 
as discussed earlier, only in the technique used; we refer here 
to forms of more direct judgment. This field constitutes the 
chief application of rating scales, and instruments of many 
varieties have been constructed. As stated earlier, there are 
scales with various degrees of refinement in the description of 
traits and of different levels of each trait. There are also 
graphic scales. The frequency of occurrence of certain kinds 
of behavior is utilized by at least one instrument ** as the 
unit for the scale. There are also a number of self-rating 
scales. 

Illustrations of two rating scales will receive separate men- 
tion. The first is Scott’s Army Rating Scale, commonly known 
as the Man-to-Man Scale." In using this rating scale, one 
thinks of persons he knows who represent the highest degree, 
the lowest degree, etc., of each quality as it is described on 
the scale, He then compares the individual to be rated with 
these selected “scale persons.” The great difficulty is in the 
first step, namely, the selection of appropriate men to rep- 


resent the scale. 


148 “Haggerty-Olson-Wickmau Behavior Rating Schedule.” Yonkers-on-Hudson, N, Y.: 
World Book Co., 1930. 

149 Percival M. Symonds, Diagnosing Pers 
The Century Co., 1931. Pp. 602. Or, see W. 
Management, pp. 203-208. A. W. Shaw Co., 


Book Co., New York.) p! 
Harold Q: ‘Ries; “Js the Rating of Human Character Practicable?” Journal of Edu- 


cational Psychology, XIL (November and December, 1921), 425-38, 485-501; XIII 
(January and February, 1922), 30-42, 81-93- 


onality and Conduct, pp. 57-61. New York: 
D. Scott and R. C. Clothier, Personnel 
1923. (Now published by McGraw-Hill 
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Another form of identifying persons with given traits is 
found in the Character Education Inquiry “Guess Who Test,” 
in which a series of character or personality descriptions is 
given for hypothetical individuals, and the pupils in a class 
are asked to write under each description the name of any 
classmate who seems to them to represent such a type of 
person."®° 

Various statistical studies have indicated that rating is sat- 
isfactory, or unsatisfactory, according to special conditions 
and factors that have entered into the particular study. Wat- 
son states that “the scale blank itself is undoubtedly less 
important than many of the conditions surrounding its use.” *°* 

Personality-rating scales and their uses are described at 
length by Symonds **? and are treated by him again in a second 
book.: Garrett and Schneck *** give much the same type of 
treatment, in briefer compass. An extensive bibliography of 
studies of ratings, with an interpretative discussion, is given 
by Watson! Cowley *** lists studies dealing with the value 
of rating scales in personnel work, Lundberg ** includes rating 
scales of several kinds which are appropriate to sociological 
studies. Hildreth *** lists rating scales which are available for 
general personality and for teacher rating. Barr and Rudisill **° 

160 Hugh Hartshorne, Mark A. May, and Julius B. Maller, Studies in Service and 
Self-Control, pp. 87-91. Studies in the Nature of Character, Tl. New York: ‘The Mac- 
millan Co., 1929. Pp. 559. Also described by Symonds, op. cit., PP. 73-4. 

151 Goodwin B. Watson, “Tests of Personality and Character,” Review of Educa- 
tional Research, p. 242, IL (June, 1932), No. 3. 

152 Percival M. Symonds, op. cit, Chap. III, “Rating Methods,” pp. 41-121. 

tee Percival M. Symonds, Psychological Diagnosis in Social Adjystment, “Rating 
Scales,” pp. 381-93; “Teacher Rating Scales,” pp. 196-204; ‘Miscellaneous Rating 
Seales,” pp. 205-6, New York: American Book Co., 1934. Pp. 362. 

i54 Henry E. Garrett and Matthew R, Schneck, Psychological Tests, Methods, and 
Results, Part Il, “Rating Scales,” pp. 103-22. New York: Harper and Bros., 1933. 

e5 Goodwin B. Watson, “Ratings, Reputation Measures,” pp. 238-45, in “Tests of 
Personality and Character,” Review of Educational Research, II (June, 1932), No. 3. 

SOW. H. Cowley, The Personnel Bibliographical Index. Columbus: Bureau of Edu- 
cational Research, Ohio State University, 1932. Pp. 433- See “Rating Scales,” p. 49, 
in the Subject Index. 

157 George A. Lundberg, Social Research, Appendix C, “Types of Social Measuring 
Devices.” New York: Longmans, Green and Co., 1929. Pp. 380. See especially pp.366-68. 

ise Gertrude H. Hildreth, A Bibliography of Mental Tests and Rating Scales, pp. 
172-5, and pp. 195-6. New York: Psychological Corporation, 1933. Pp. 242. 

Tee A. S. Barr and Mabel Rudisill, An Annotated Bibliography on the Methodology of 


Scientific Research as Applied to Education, pp. 74-81. Bulletin of the Bureau of Edu- 
cational Research, No. 13. Madison. Wis.: University of Wisconsin, June, 1931. Pp, 129. 
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list a number of studies and applications of rating scales. The 
Psychological Bulletin includes rating scales in its annual re- 
view of tests, which has appeared every year since 1926 
except in 1931. 

As in other cases, the best material on the preparation of 
ating scales is found in the treatises which describe them 
and their applications. The student should consult especially 
the reviews in the Psychological Bulletin and in the Review 
of Educational Research, as referred to. Treatises by Symonds 
and by Garrett and Schneck discuss construction and give 
additional references. The problems of constructing rating 
scales are primarily the difficulty of securing satisfactory cate- 
gories, and of setting up criteria by which the satisfactoriness 
of the scale may be carefully determined. One must have 
categories that possess approximately the same significance to 
all who will use the rating scale, and this is not easy to attain. 
Further, the categories must fit together into a satisfactory 
pattern so that, when taken together, they will afford an ade 
quate picture. Some workers emphasize the statistical aspects 
of scaling; this point of view is represented in a report by 
Richardson and Kuder." 

Score cards. Score cards as a group represent the most 
elaborate form of rating instrument for utilizing judgments 
directly. There is no sharp dividing line between rating scales 
and score cards, and many workers use the two terms inter- 
changeably.% In the present treatment, however, a distinc- 
tion is made. The typical score card provides for more aspects 
of the object to be rated and has a defnite number of “points” 
to be allotted to each item. A rating scale has fewer charac- 
teristics and uses a variety of different means of scaling, some 
of which are quite indefinite. Further, a rating scale may have 
descriptions of various degrees of each trait or characteristic ; 


160 Fe ample, see Psychol, ical Bulletin, XXXII (July, 1935), 509-10. 

161 MW. Richardson and G F. Kuder, “Making a Rating Scale That Measures,” 

Personnel Journal, XII (June, 1933), 36-40- 
162 In the United States Office of Education 

catwn and in the Education Index, the topics are 

either topic may include material for both. 


Bibliography of Research Studies in Edu- 
likely to be cross referenced, and 
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a score card may have general standards or criteria, printed 
separately, at some length, and describing only a satisfactory 
or ideal level. The score card differs from the check-list pri- 
marily in calling for an evaluation of each aspect rather than 
noting merely its presence or absence. 

The most common application of the score card has been 
in the field of rating school buildings. To illustrate, we may 
use the Strayer-Engelhardt Score Card for High-School Build- 
ings..** This divides the school plant into seven large divisions, 
such as site, general aspects, service systems, classrooms, etc. 
A total of 1,000 points is apportioned to the seven divisions 
according to their judged importance. Each division is then 
broken down into several sub-divisions; for example, the site 
is divided into: (A) location, (B) nature and condition, and 
(C) size and form. These are in turn broken down into still 
smaller aspects, with an indication of the number of points 
allowable for each aspect. 

With a moderate amount of training, one can learn what 
constitutes desirable standards on each of these small aspects 
and can rate each part of a building with considerable fair- 
ness. A manual of standards is available for ajding the rater, 
and forms for making field notes have been prepared and are 
commercially available. 

The important features of this procedure are that one re- 
acts to a single unit of the gross situation at a time, he has 
been given mental standards with which he can compare this 
unit, and the number of points that he can allow for this as- 
pect is assigned. If one should go far wrong on a few phases, 
this error of judgment would not invalidate his final outcome 
to any large extent, unless he tended to be wrong in the same 
direction in most of his detailed judgments. The score card 
being discussed’ provides for 159 separate specific judg- 


163 Published by the Bureau of Publications, Teachers College, Columbia University, 
New York. 

164 The use of this particular score card for illustration should not lead to the im- 
plication that it is the only one. There are a number of score cards for school buildings 
of various kinds, Different ones deal with elementary, junior high, and high-school build- 
ings; normal schools and teachers colleges; city and rural school buildings; and the 
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ments; the very number and variety of aspects that are to be 
judged tend to prevent subjective bias in the composite score. 
Even with such safeguards, however, the authors caution that 
it is desirable to have at least three individuals score each 
building, and to use their average score. 

As in other cases where measurement utilizes units that are 
essentially subjective and highly variable in what they repre- 
sent, interpretation of the total score presents a problem. It 
is possible to divide up the scale somewhat arbitrarily and 
to suggest that a total score of more than three-fourths of 
the maximum (750-1000 on the Strayer-Engelhardt scales) 
represents a good building; from one-half to three-fourths 
(500-750) a fair building; from one-fourth to one-half, a poor 
building; and below one-fourth of the maximum score, an 
unsatisfactory building. In another place *°* Engelhardt points 
out that a score of 900-1000 represents a “highly satisfactory 
degree of construction and equipment.” A rating of 700-900 
is fairly satisfactory, but the building should be studied for 
the possibility of improving the parts of it which the score 
card shows to be most unsatisfactory. A score of 600-700 
indicates that considerable alteration is needed ; buildings re- 
ceiving a score of 500-600 require extensive repairs and re- 
placements in order to be reasonably usable. Where a building 
scores below 500, it should be abandoned as soon as possible 
and no funds spent trying to make it more satisfactory, since 
too much money would be required. 

It will be seen that the foregoing interpretation rests upon 
the association of certain scores with general judgment and 


administration building. References to seventeen such score cards may be found in: 
“School Buildings, Grounds, Equipment, Apparatus, and Supplies,” Review of Educa- 
tional Research, II (December, 1932), No. 5; see bibliographical references, Nos. 1, 5, 
15, 36, 64, 70, 71, 81, 82, 83, 84, 85, 86, 152, 172, 194, 204. Other references deal 
with standards for these cards. Three additional score cards are listed in: Henry Lester 
Smith and Leo Martin Chamberlain, A Bibliography of School Buildings, Grounds, and 
Equipment, Bulletin of the School of Education, Indiana University, Vol. IV (January, 
1928), No. 3. Bloomington, Ind.: Bureau of Codperative Research. Pp. 326. See refer- 
ences, Nos. 314, 321, 323. 

165 N. L. Engelhardt, A School Building Program for Cities, p. 57. Teachers College 
Contributions to Education, No. 96. New York: Teachers* College, Columbia University, 


1918, Pp. 130. 
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general experience concerning the buildings; it is, however, 
about the only kind of interpretation which is possible; and 
if confirmed by many years of experience, it is to be regarded 
as altogether satisfactory. Although he agrees in general with 
the above interpretation, T. C. Holy +% points out that “it does 
not follow that two buildings scoring 500 each are of equal 
value.” This comes about, of course, because the scores can 
be made up in different ways. It should follow, however, if 
the rating scales were ideal, that each of two buildings with 
the same score would yield the same gross amount of satis- 
faction in use, although not necessarily identical in the forms 
of satisfaction. 

The score-card technique has had a number of additional 
applications to subjects closely related to buildings. For ex- 
ample, score cards have been prepared for the selection of 
school-building sites,!°* for architectural service and building 
plans,** for school-building utilization,’*® for the school hous- 
ing program,’ for structural and housekeeping sanitation,’’* 


166 T. C. Holy, “Technics Used in School Building Surveys,” Chapter VII, p. 53, in 
“Methods and Technics of Educational Research,” Review of Educational Research, IV 
(February, 1934), No. 1. 

167N. L. Engelhardt and W. B. Featherstone, ‘Score Card for Selection of School 
Sites,” in American School and University, 1930-31, Vol. III, pp. 29-31. New York: 
American School Publishing Corporation, 1930. 

“Score Card to Be Used in the Selection of School Building Sites.” New York: 
Teachers College, Columbia University, 1929. Pp. 4. 

L. W. Reese, “A Score Card for Selecting the Site for a Consolidated School,” 
American School Board Journal, LXX (January, 1925), 79-80, 130. 

Robert Earl Iffert, “A Study of School Building Sites in Certain Districts of 
Allegheny County.” Master’s thesis. Pittsburgh: University of Pittsburgh, 1927. Pp. 118. 

168 Z. M. Smith, “A Score Card for Evaluating Architectural Service in the Planning 
and Constructing of Rural School Buildings.” Indianapolis: State Department of Public 
Instruction. 

George D, Strayer, N. L. Engelhardt, and F. W. Hart, ‘‘Schoolhousing Series.” School- 
housing Form No. II, “Checking Chart and Score Card for School Building Plans and 
Specifications.” (For Use by State and Local Building Authorities.) Albany, N. Y.: 
C. F. Williams and Son, Inc., 1920. 

169 Edgar L. Morphet, The Measurement and Interpretation of School Building 
Utilization. Teachers College Contributions to Education, No. 264. New York: Teachers 
College, Columbia University, 1927. Pp. 102. 

George D. Strayer, N. L. Engelhardt, and Edgar L. Morphet, School Building Utiliza- 
tion Forms. New York: Teachers College, Columbia University, 1927. Pp. 32. 

170 George D. Strayer and N. L. Engelhardt, A Checking List for Administrative 
Policies and Programs Concerned with Schoolhousing (Tentative Outline). New York: 
Teachers College, Columbia University, 1926. Pp. 36. 

171 Walter S. Cornell, “A System of Sanitary School Inspection with Graded Stand- 
ards and Exact Ratings of School Buildings,” American School Hygiene Association. 
Proceedings of Eighth Congress, 1915, pp. 44-9. 
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for janitorial services,'* and for school seating." These ref- 
erences will indicate the extent to which the device has been 
developed in connection with school housing and educational 
equipment. 

Clark’s score card for the school budget, Strayer and 
Engelhardt’s score card for school records and reports,’ 
Proctor’s score card for guidance,’ Harrington’s score card 
for the organization of junior high schools," and Moehlman’s 
score cards for publicity material and for the parent-teacher 
association,’*® illustrate other applications of score cards to 
administrative problems. Mort and Hilleboe have prepared a 
score card for elementary-school practices.1”° The Committee 
on Revision of Standards for Higher Institutions, of the North 
Central Association, validated a score card for aid in apprais- 
ing colleges.’*° The work of the joint Secondary School Stand- 
ards Committee +° may result in the use of a score card or 


rating scale as an important means in determining the accredit- 
ing of high schools. Score cards are also used for rating the 


472 N. L. Engelhardt, C. E. Reeves, and G. F. Womrath, “Score Card for Public 
School Janitorial-Engineering Service.” New York: Teachers College, Columbia Uni- 
versity, 1926. Pp. 6. 

17a C. A. Anderson, “Tentative Score Card for Elementary Desks and Seats,” Ameri- 
can School Board Journal, LXXIX (July, 1924), 46-7. 

174 Harold F. Clark, “Suggestions for Scoring School Budgets,” American School 
Board Journal, LXXI (October, 1925), 47-8, 133-4. 

175 George D. Strayer and N. L. Engelhardt, A Score Card and Standards jor the 
Records and Reports of City School Systems. New York: Teachers College, Columbia 
University, 1923. Pp. 81. 

176 William M. Proctor, “Evaluating Guidance Activities in High Schools,” Voca- 
tional Guidance Magazine, IX (November, 1930), 58-66. 

i77 Harold L. Harrington, “A Score Card for the Mechanical Organization of Junior 
High Schools,” Doctor’s thesis, University of Michigan, 1931. Pp. 292. Also in 
Abstracts of Dissertations and Theses in Education 1917-31, PP- 70-3; Bureau of Edu- 
cational Reference -and Research, Monograph No. 1. Ann Arbor, Mich.: School of 
Education, University of Michigan, 1932. 

178 Arthur B. Moehlman, Public School Relations. Chicago: Rand, McNally and Co., 
1927. Pp. 262. In Chapter XVII, “Appraisal,” Moehlman gives score cards for the 
annual report, the house organ or ‘teachers bulletin, the home-contact bulletin, and the 
parent-teacher association. 

179 Paul R. Mort, “Rating Scale for Elementary-School Organization— Development," 
Teachers College Record, XXXII (October, 1930), 34-49. | i 

180 John Dale Russell, ‘Measuring Excellence of Institutions of Higher Education,” 
pp. 83-94, in Twenty-Second Annual Conference on Educational Measurements. Bulletin 
of the School of Education, Indiana University, XII (December, 1935), No, 1 
Bloomington, Ind.: Bureau of Codperative Research. Pp. 97. See especially pp. 90-91. 


181 See references in footnote 292- 
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homes of pupils,®? and for appraising other institutions and 
community conditions.%* In this connection, the ~eader may 
consult the later discussion on “The Appraisal of Educational 
Service.” 

x. Score cards in the selection of textbooks. Another field in 
which the use of score cards is becoming common is that of 
evaluating textbooks for adoption. The selection of textbooks 
constitutes a problem of large importance, not only because 
considerable sums of money are involved, but because the text- 
book is, in the hands of many teachers, a more potent influence 
in determining what is taught than is the course of study which 
has been officially adopted. The literature on the selection of 
textbooks is already extensive.** The field of rating and selec- 
tion is distinct from that of textbook analysis, which was re- 
ferred to in the section, “Documentary Frequency Studies,” 
in Chapter VII, although rating may well include a number of 
the elements revealed by textbook analyses.**® 


182 Percival M. Symonds, Diagnosing Personality and Conduct, pp. 546-9. New York: 
The Century Co., 1931. Pp. 602. 

—, Psychological Diagnosis in Social Adjustment, Chapter XIII, “Environment 
Scales and Tests,” pp. 328-32. New York: American Book Co., 1934. Pp. 362. 

See also pp. 265-77 of the reference in the following footnote. 

188 George A. Lundberg, Social Research, Chapter X, “The Measurement of Social 
Institutions,” pp. 243-99, and Appendix C, “Types of Social Measuring Devices,” pp. 
362-6. New York: Longmans, Green and Co., 1929. Pp. 380. 

Ellery Reed, “A Scoring System for the Evaluation of Social Case Work,” Social 
Service Review, V (June, 1931), 214-36. 

184 The Textbook in American Education, Chapters VII, VIII, XIV, XV, XVI, and 
XVIII. Thirtieth Yearbook of the National Society for the Study of Education, Part II. 
Bloomington, Ill.: Public School Publishing Co., 1931. 

Frank A. Jensen, Current Procedure in Selecting Textbooks. Philadelphia: J. B, Lip- 
pincott Co., 1931. Pp. 157. 

Florence D. Fuller, Scientific Evaluation of Textbooks. Boston: Houghton Mifflin Co., 
1928. Pp. 88. 

Bibliographies may be found in the following: 91 selected and annotated references 
are given in Chapter XVIII of the Thirtieth Yearbook of the National Society for the 
Study of Education, cited in the present footnote. Jensen gives an important bibliog- 
raphy (pp. 151-4). 

The Curriculum Laboratory of Western Reserve University, Cleveland, has issued: 
“Bibliography: How to Select a Textbook,” Bulletin No. 24, 1932. Pp. 3 (mimeo- 
graphed). See also “Evaluation of Courses of Study and Textbooks,” Chapter III, in 
“The Curriculum,” Review of Educational Research, IV (April, 1934). 

For current material see “Textbooks” in the Annual Bibliography of Research Studies 
in Education, of the United States Office of Education, and ‘Textbooks—Rating”’ and 
“Textbooks—Selection” in the Education Index. 

185 See for example, Ivan R, Waterman and Irving R. Melbo, “Evaluation of Spelling 
Textbooks,” Elementary School Journal, XXXVI (September, 1935), 44-52. Various 
analyses are reported and used as bases for rating. 
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To illustrate the content of the textbook score card, attention 
will be focused on one of them. Maxwell +° presents an elab- 
orate score card worked out by Mildred Dawson, for the rating 
of textbooks in language. This covers six pages in print. Its 
four main divisions are: I. Mechanical features, II. Derivation 
of materials, III. Content, IV. Presentation. Each of these is 
broken down into principal and minor subdivisions, making 
in all seventy-five separate aspects that are to be rated. The 
«basis of evaluation” is given, such as “Inspection,” “Sampling 
of alternate so-page groups,” etc. Next, the “Method of 
Evaluation” is given, such as “Rank,” “Rate from Exceptional 
to Poor,” “Number of instances per page,” etc. The next col- 
umn on the blank is for recording the points allotted, and 
finally a general rating of A, B, C, or D is given to a book, 
in each division, on the basis of the total ratings in that di- 
vision. Decision is finally made on the basis of the separate 
ratings on each book, rather than on the basis of a numerical 
total score. The score card is accompanied by directions and 
explanations. 

Jensen *** finds that, of 172 cities studied, about 40 per cent 
use score cards in selecting textbooks, and that the instruments 
are constructed by some one in the local system (textbook 
committee, research bureau, supervisor, etc.) in about two- 
thirds of the cases. Criticisms of the use of score cards in the 
selection of textbooks seem to grow primarily out of the fact 
that there is a strong tendency to set up such instruments with 
a particular set of books in mind, so that the score cards 
actually favor those books and thus result in the selection of 
books which would have been chosen anyway without the use 
of a score card. Score cards prepared by book publishers 
suffer the same criticism in a more pronounced form, A second 

186 C. R. Maxwell, “The Use of Score Cards in Evaluating Textbooks,” Chapter. VITE 


in The Textbook in American Education. ‘Thirtieth Yearbook of the National Society 
for the Study of Education, Part II. Bloomington, Il.: Public School Publishing Co., 
1931. 

187 Frank A. Jensen, Current Procedure in Selecting Textbooks, Chapter VI, “Devices 
for Evaluating Textbooks,” pp. 126-45- Philadelphia: J. B. Lippincott Co., 1931. 
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basis of criticism is that score cards are rather mechanical and 
overlook a number of the more important psychological ele- 
ments of a textbook which can be evaluated only by actual 
tryouts or general appraisal. Whether these are inherent de- 
fects is debatable. 

Some further comments on the evaluation of textbooks are 
given in Chapter XIV. The increasing use of score cards in this 
and other fields and the two major difficulties just mentioned 
indicate that this is an important area in which research of 
considerable significance may be conducted. The charac- 
teristics of a satisfactory score card, and the best techniques 
for constructing one, need to be determined. They will probably 
be found to vary with the field and purpose. 

On the construction of score cards in the field of school 
buildings there is only meager literature. Commenting in 1916 
on the Score Card for City School Buildings, Strayer *** ex- 
plains that it was developed through two years and a summer 
session with five groups of graduate students, and that the 
weights were assigned to the large divisions first, then to sub- 
divisions, and then to the elements, by taking the medians of 
the ratings of 200 graduate students. Engelhardt, in com- 
menting upon this same score card, says that it “is constructed 
on the same plan as those utilized during past years in the work 
of agricultural colleges.” He, further, gives statistical evidence 
to the effect that the median rating of three judges correlated 
to the extent of .98 with the median rating of five judges, indi- 
cating that rating by three people was probably satisfactory.” 
One will find more adequate descriptions by Clark," 


188 George D. Strayer, “Score Card for City School Buildings,” Chapter III, pp. 41-51, 
Fifteenth Yearbook of the National Society for the Study of Education, Part I. Chicago: 
University of Chicago Press, 1916. Pp. 172. 

189 N. L. Engelhardt, A School Building Program for Cities, pp. 54-6. Teachers Col- 
lege Contributions to Education, No. 96. New York: Teachers College, Columbia Uni- 
versity, 1918. Pp, 130. 

190 Ibid., p. 60. 

191 Harold F. Clark, “Measuring the Budgetary Procedure of a School System,” 
pp. 20-7, in the Eleventh Conference on Educational Measurements. Bulletin of the 
School of Education, Indiana University, I (January, 1925), No. 3. Bloomington, Ind.: 
Bureau of Coöperative Research. Pp. 141. See also the American School Board Jour~al, 
LXVIII (June, 1924), 47-8, 133-4. 
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Senska,? Cox, and Marye; *** other suggestions on con- 
struction may be gleaned from certain references already 
given. 

Underlying assumptions. Two principles, on which appraisal 
instruments are based, represent fundamental assumptions. 
First, it is assumed that better judgment can be secured on the 
significant aspects of an object (or situation) by centering 
attention on one aspect at a time. This is a principle which is 
substantiated by common experience, in so far as we are think- 
ing of the vividness of a single element in the focus of atten- 
tion, in contrast’ to its clearness when attention is spread 
indifferently over a group of undifferentiated elements. There 
are other aspects of the premise, however, which are not so 
definitely established. Question arises concerning the isolation 
of elements which are the crucial ones. It is not difficult to pick 
out certain aspects of any phenomenon (object or situation) 
and concentrate upon them, but practical workers who have 
developed good judgment concerning complex situations are 
likely to have a feeling that the aspects selected by the makers 
of observation and appraisal schedules omit certain intangibles 
that are crucial. The question is important ; it will undoubtedly 
be answered more satisfactorily as our research advances and 
becomes increasingly refined. 

The second fundamental assumption involved in all of the 
instruments which yield a general total or composite rating 
is that general value can be approximated by a summation 
of the values of parts. Even granting that the first question 
has been satisfactorily answered, and that all of the really 


192 N. M. Senska, “Making a Detailed Score Card for Grading Student Teachers,” 
Educational Administration and Supervision, XI (March, 1925), 191-201. 

198 Rose M. Cox, “The Development of Score Cards for Evaluating Silent Reading 
Lessons in the Intermediate Grades,” Teachers College Journal (Indiana State Teachers 
College, Muncie), II (July, 1932), 391-22. Also, her Doctor's thesis, at Indiana 
University, “The Development of Score Cards for Evaluating Silent Reading Lessons in 
the Intermediate Grades,” 1929. Pp. 344. 

194 Mary E. Marye, “A Form for Rating Textbooks in English Composition Prepared 
for the oth and roth Grades,” School Review, XXXVIIT (February, 1930), 124-37. 
Described briefly in ‘Special Methods on the High School Level,” Review of Educa- 
tional Research, II (February, 1932), 36-7. Aiso her Master’s thesis at the University 
of Chicago in 1928, “The Construction of a Rating Form for Evaluating Composition 


Textbooks.” Pp. 275. 
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significant aspects have been included in the rating, there is 
still room for question that any mathematical functions (such 
as a weighted sum) of these separate ratings will yield a rela- 
tive value that corresponds well with one’s reaction to the 
object or situation as a whole. We have involved here the 
question of atomistic and organic conceptions in analysis,’°° 
and the challenges raised by Gestalt psychology as to whether 
something is not lost when a complex whole is divided into 
parts—at least some element of pattern. Perhaps this element 
could be restored if the proper method of recombining ele- 
mental aspects could be discovered; so far we have been con- 
tent with thinking of the problem in terms of weighted 
summation, which may prove too simple an approach. This sec- 
ond question also forces a new interpretation of the first one, 
for the aspects (elements) must be significant not only when 
viewed independently but when viewed in the light of their 
contribution to the picture as a whole.” 

From the practical standpoint, these questions are not so 
acute. In the first place, improvements in practice or in ob- 
jects (such as school buildings) are for the most part made in 
terms of details. We give attention to one relatively small area 
at a time and seek to improve it. The use of appraisal instru- 
ments is in line with this normal procedure. In the second 
place, the instruments are not likely to be applied at any one 
time to the appraisal of objects or practices which vary funda- 
mentally. That is, a score card for buildings would not be 
applied to some experimental building that was constructed to 
suit an entirely special philosophy of education, or that was 
built on some new theory of building construction which sought 
to make an epochal advance, Appraisal instruments are clearly 
not adapted to the evaluation of fundamentally different 
things, unless they are specially constructed for that purpose. 


195 Ernest R. Mowrer, Domestic Discord, Chapter II, ‘“Case-Work and the Atomistic 
Conception,” pp. 14-22. Chicago: University of Chicago Press, 1928. Pp. 277. 

196 Raymond Holder Wheeler, ‘‘Postulates for a Theory of Education: I, a 
Methodology for Educational Research,” Journal of Educational Research, XXIX (No- 
wember, 1935), 187-95. 
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In the third place, it is possible to include in any schedule 
elements of varying generality, some very detailed, some rather 
broad, so that a hierarchy of levels is represented. The question 
of losing elements of organization or pattern through dividing 
things is thus met, at least in part. Perhaps more attention 
to this practical possibility should be given in the construction 
of schedules. It will, of course, be recognized that even without 
giving it specific attention there is little equality of generality 
in the items which are included; this is probably to be regarded 
as an asset. In the fourth place, it is probable that administra- 
tive action will always be tempered by general opinion and 
common sense—sometimes to the exasperation of the technical 
research worker, but not always to the detriment of education. 
As long as these elements of general judgment will enter into 
practical decisions, the extent to which they may be lacking 
in the instruments is not serious, though such a lack does tend 
to discredit the contributions of research in the eyes of the 
practical worker. 

_ The foregoing remarks concerning the practical situation 
have no implication of the ideal situation. But we must always 
temporize ideals with practical considerations. Perhaps an ideal 
instrument would be so cumbersome and expensive of effort 
and time that its use would not be warranted. Possibly a com- 
bination of ratings and general judgment will ultimately be 
found to be more desirable than the scientist’s dream of a per- 
fect instrument. After all, appraisal schedules are normative 
instruments; they reflect general tendencies, tempered by the 
superior and by the inferior, but they represent in the main 
the things we are most used to. Differences in objects or prac- 
tices which involve questions of changing philosophies and 
points of view must be evaluated by other means—principally 
by human beings giving their best attention directly to the 
consideration of requirements and consequences. The problem 
of human values may be studied scientifically, but it can never 
be supplanted by science without our having surrendered our 


prerogatives as human beings. 
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B. Appraisal procedures which are relatively objective. 
This is the second division concerned with appraisal proce- 
dures. Two groups of studies will be discussed, namely, those 
employing index numbers, and school surveys. A third topic 
is concerned with the selection of aspects cf educational insti- 
tutions for indicating quality of service. 

In referring to these procedures as relatively objective, one 
should not lose sight of the fact that they are based on judg- 
ments or human values. That is, if an index number is to repre- 
sent worth or quality, it must contain those elements which 
human beings have come to regard as valuable. The selec- 
tion of these elements, and the securing of quantitative meas- 
ures of them, are critical steps in index-number work. Again, 
in the school survey, it is necessary to devote considerable 
attention to the discovery of the vital problems in a school 
system, and to the identification of the factors which are con- 
tributing to these. Such steps require judgment—aided by such 
objective evidence as is obtainable, but initially and finally it 
is judgment that must be relied upon, The procedures are con- 
cerned with objective data only in an immediate sense. 

Index numbers, In the desire to make complex comparisons 
as objective as possible, educators have often turned to the use 
of index numbers. These represent a particular form of an- 
alyzing quantitative data which can be gathered by one means 
or another, usually from available reports. In the typical case, 
index numbers do not depend for their basic figures on judg- 
ment at all; after the characteristics which are to be reflected 
in the index number and their respective weighting have been 
decided upon, the rest of the procedure is usually objective. 

An index number is essentially an average. It combines in 
one figure the average of a number of different factors (variable 
elements). It is, however, an average of ratios, each of the fac- 
tors being expressed as a per cent of its value at some other 
time (or perhaps at some other place). The technique is, there- 
fore, applicable whenever it is desired to represent in a single 
figure the average status of a number of factors, each expressed 
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as a variation from a particular set of values that has been 
accepted as a base. The selection of a base year or a base 
locality, with its attendant value for each factor, though not 
entirely without effect on the comparisons, is not a crucial 
problem and may be done arbitrarily to suit convenience. 

Index numbers have been applied to a number of phases of 
education, most of which are administrative. Most ambitious 
of these applications have been the attempts to rate the educa- 
tional activities of the various states. First among these was 
a study by Ayres in 1912,1** which received little publicity. 
In 1920, however, Ayres published his Jndex Number for State 
School Systems,” which became widely known and much dis- 
cussed. This publication showed the standing of each of the 
states in terms of the index number, for 1890, 1900, 1910, and 
1918. Indexes for the United States as a whole were printed 
for each of the forty-eight years from 1871 to 1918. The index 
was based upon ten items of information taken from the sta- 
tistical reports of the United States Office of Education. These 
items, or factors, read: “Per cent of school population attend- 
ing school daily,” “Average number of days schools were kept 
open,” “Average expenditure per child in average attendance,” 
“Expenditure per teacher employed, for salaries,” etc. 

Phillips has made an extension of the Ayres index,'*® carry- 
ing it forward, modifying it slightly, and offering supple- 
mentary data. Schrammel prepared a third index number for 
the states.2°° In addition to these state index numbers, there 
have been at least three others prepared for cities, and four 


197 L. P. Ayres, A Comparative Study of the Public School Systems in the Forty- 
Eight States. New York: Russell Sage Foundation, 1912. Pp. 33. 

198 L. P. Ayres, An Index Number for State School Systems. New York: Russell Sage 
Foundation, 1920. Pp. 70. 3 

199 Frank M. illips, “Educational Rank of the States, 1930,” American School 
Board Journal, LXXXIV (February through May, 1932). (Also available as a forty- 
page reprint, from the author, Washington, D. C.) Earlier articles: “Educational Rank 
of the States, 1924,” American School Board Journal, LXXII (April, 1926), 47, 141; 
and “Educational Ranking of the States by Two Method a American School Board 
Journal, LXIX (December, 1924), 47-9. This earliest se ies is available as a 32-page 
publication from the Bruce Publishing Co., Milwaukee, Wisconsin, 1925. J 

200 Henry E. Schrammel, The Organization of State Departments of Education, Chap- 
ter IX, “Ranking of States According to Educational Achievements, PP: 115-34. 
Bureau of Educational Research Monographs, No. 6. Columbus, Ohio: Ohio State Uni- 


versity Press, 1926. 
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for counties. This whole field has been excellently reviewed 
and summarizd by the Research Division of the National 
Education Association,” with a bibliography of 47 references 
added. The same publication contains data for the states on 
five factors selected by the Research Division, which refrains 
from combining these into a single index because it does not 
wish to decide on the relative weighting of each factor. An 
index number for higher education in the various states, based 
on eight factors, has been prepared by Chamberlain and 
Meece.2” Private and public education are considered sep- 
arately and jointly. 

Numerous other series of data for states have been pub- 
lished, in ratio form, but it is not appropriate to regard every 
series of ratios as index numbers. For example, one may calcu- 
late and perhaps plot on a graph trends showing the change in 
school enrolment and the change in school-building con- 
struction during the last ten or twenty years, expressing each 
of these series in terms of per cent of the initial year values.?°* 
But such series are simply figures turned into per cents; they 
constitute single elements that might, along with a number of 
other such elements, enter into an index number. An index 
number is an average of a number of such series, taken at any 
desired points (such as years, or places). Thus, Witham, who 
used only holding power of the schools,?°* and Sias, who used 
only the average number of days of attendance for each child 
of school age,” as indexes of the efficiency of state school 

201 “Estimating State School Efficiency,” Research Bulletin of the National Educa- 
tion Association, X (May, 1932), 104-12. 

See also, for some additional references: Frank L. Shaw, State School Reports, pp. 
103-7. Teachers College Contributions to Education, No. 242. New York: Teachers Col- 
lege, Columbia University, 1926. Pp. 142. 

202 Leo M. Chamberlain and L. E. Meece, State Performance in Higher Education. 
Bulletin of the Bureau of School Service, V (March, 1933), No. 3. Lexington, Ky.: 
University of Kentucky. Pp. 37. 

203 See for illustration, “The Nation’s School Building Needs,” Research Bulletin of 
the National Education Association, XIII (January, 1935), No. 1. Figure II on p. 7, 
and Figure VII on p. 27. 

204 Ernest C. Witham, “Index of Holding Power,” American Educational Digest, 
XLVI (August, 1927), 548-51. 

, “Public School Progress of the States,” American School Board Journal, 
LXXV (October, 19273, 37-9. 


205 A. B. Sias, “The Financing of a State School System.” Doctor’s thesis, Stanford 
University, 1926. 
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systems, cannot appropriately be said to have employed index 
numbers in their analyses.*°* 

Again, an index number is based on a sampling of items 
rather than on totals which are complete. Thus the total ex- 
penditures for school systems, although representing a di- 
versity of elements, are not an appropriate basis for index num- 
bers because they are complete, rather than being a sampling 
of costs. Such totals may be expressed as ratios and called 
indexes, and they may be of great value,” but they lie outside 
the strict limits of the present category. In some cases it is not 
readily possible to determine whether an index should be re- 
garded as truly an index number or not. 

It may be appropriate to call attention to certain index num- 
bers which have been devised in the field of sociology to meas- 
ure changes in complex social conditions. These are referred 
to in the footnote.: 

Index numbers were originally constructed to measure 
changes in prices of commodities. Although such indexes do not 
serve directly the purpose of rating, they make possible critical 
comparisons which are closely associated with appraisal. There 
have been a number of price indexes constructed in the field 
of education. Notable among these were three series of index 
numbers prepared by H. F. Clark **° and associates, dealing 


206 For definitions of index numbers, see Irving Fisher, The Making of Index Num- 
bers, Chapter I, “Introduction,” pp. 1-10. Boston: Houghton Mifflin Co., 1922 and 1927. 
Pp. 538. 

‘Also, Willford I. King, Index Numbers Elucidated, Chapter III, “The Nature of 
Index Numbers,” pp. 46-9. New York: Longmans, Green and Co., 1930. Pp. 226, See 
the comments of Allyn A. Young, Handbook of Mathematical Statistics, edited by H. L. 
Rietz, p. 181. Boston: Houghton Mifflin Co., 1924. Pp. 221. 

207 For example, see John K. Norton, The Ability of the States to Support Education, 
Washington: National Education Association, 1926. Pp. 88. 

208 George A. Lundberg, Social Research. New York: Longmans, Green and Co., 1929. 
Pp. 380. In Appendix C, p. 362, the following are listed: 

I. M. Rubinow, “A Dependency Index for New York City,” American Economic 
Review, VIII (December, 1918), 713-40. 

F. S. Chapin, “Dependence Indexes for Minneapolis: A New Method in Seasonal and 
Cyclical Analysis,” Social Forces, V (December, 1926), 215-24. 

R. G. Hurlin, “Indexes of Family Case Work Loads,” The Survey, LIX (February, 
1928), 634-5. 

See also footnote 288. 

200 The preparation and use of these three series are described in the following publi- 


cation: Harold F, Clark, Index Numbérs in School Administration. Bulletin of the School 
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with the cost of school supplies,™? the price (interest rate) for 
school bonds,™™ and the cost of school buildings.” The latter 
field has also been worked on by others,” the most outstand- 
ing study being that by Burgess.*** The cost of equipping new 
buildings has been studied by Loomis,” utilizing index num- 
bers. Davis employed an index-number technique to study the 
increasing cost of operating school buildings.?*° 

These data on educational building costs may be supple- 
mented by figures on general building costs. The United States 
Bureau of Labor Statistics gives index numbers on the cost of 
building materials *7 and rates of union labor.*!* Other figures 


of Education, Indiana University, IIT (January, 1927), No. 3. Bloomington, Ind.: 
Bureau of Codperative Research, August, 1928. Pp. 35. 

See also, by the same author: “Index Numbers in Educational Work,” Teachers Col- 
lege Record, XXX (February, 1929), 453-60, and “Index Numbers of Costs in Educa- 
tion,” March-May, 1929. 

210 Harold F. Clark and John Guy Fowlkes, “Index Numbers for School Supply 
Prices.” Appeared monthly in the Nation's Schools from September, 1928 (Vol. II) to 
December, 1929 (Vol. IV) and was then combined with the index of school-building 
costs, being discontinued with the March, 1930, issue (Vol. V). 

211 Harold F. Clark, “Index of School-Bond Prices.” Appeared monthly in the 
American School Board Journal from January, 1928 (Vol. LXXVI) to November, 1931 
(Vol, LXXXIII). 

212 Harold F. Clark, This series began as ‘School Building Cost Index” in American 
Education Digest, XLVIII (December, 1928), 28. Continued as “School Building 
Index” in School Executives Magazine from January to August, 1929 (Vol. XLVIII); 
continued with Oscar K. Buros, joint author, as “Index of School Building Prices,” in 
the same magazine, September to December, 1929 (Vol. XLIX); then combined with 
the index of school supply prices in the Nation's Schools from January to March, 1930 
(Vol. V). 

218 T. C. Holy and W. E. Arnold, “School Building Expenditures in Relation to 
School Building Costs,” American School Board Journal, LXXXV (July, 1932), 41-2. 
A. C, Monahan, “notes,” American. School Board Journal, LXIX (October, 1924), 64; 
LXXI (September, 1925), 65; and LXXXIII (August, 1931), 74. 

214 Randolph W. Burgess, Trends of School Costs. New York: Russell Sage Founda- 
tion, 1920. Pp. 142, Chapter V traces building costs from 1841 to 1920, in terms of 
index numbers. 

Aiso given in part in the following: ‘“Eighty-year Fluctuations in the Cost of American 
School Buildings,” American School Board Journal, LXII (January, 1921), 57-8; also 
in Proceedings and Addresses of the National Education Association, LVIII (1920), 329- 
30; also in School Life, V (August 15, 1920), 11-12. 

215 Arthur K, Loomis, The Technique of Estimating School Equipment Costs. Teachers 
College Contributions to Education, No, 208. New York: Teachers College, Columbia 
University, 1926. Pp. 112. Also, School Equipment Costs. Teachers College, Columbia 
University, 1926. Pp. 259. 

216 Harvey H. Davis, “An Index of School Plant Operation Costs,” American School 
Board Journal, LXXIII (July, 1926), 53. 

217 Bulletin No. 493 (see footnote 221), p. 243, gives index numbers on “Changes 
in Buying Power of the Dollar” from 1913 to 1928, for lumber, brick, cement, steel, 
paint materials, and other building materials, separately and combined. 

218 “Index Numbers of Wages Per Hour, 1840 to 1932 (Exclusive of Agriculture),” 
Monthly Labor Review, XXXVII (September, 1933), 632. 

Each issue of the magazine usually carries some data on wage rates and hours. See 
alse the U. S. Department of Commerce Statistical Abstract of the United States. ' 
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and indexes are also available. Chawner *!° refers to a number 
of such indexes, setting forth certain cautions that should be 
observed in their use. 

1. Index numbers of economic conditions. The U. S. Bureau 
of Labor Statistics calculates three series of general index 
numbers which are of basic importance. They cover the cost 
of living, wholesale prices, and retail prices. The index numbers 
dealing with cost of living °° are perhaps the most widely 
known in educational circles. They extend back to 1913, by 
months, and are available separately for each of six groups of 
commodities, namely, food, clothing, housing, fuel and light, 
house furnishings, and miscellaneous. They are also available 
separately for a number of individual cities. 

The series on monthly wholesale prices *** extend back to 
1890, in considerable detail, and are available as annual aver- 
ages back to r8or. Other early index numbers of wholesale 
prices are referred to by Warren and Pearson,**? who have 
calculated their own series running back to 1786.2** Index 
numbers of wholesale prices are sometimes supplemented by 


219 Lowell J. Chawner, “Construction Cost Indexes as Influenced by Technological 
Change and Other Factors,” Journal of the American Statistical Association, XXX 
(September, 1935), 561-76. 

220 Cost-of-living index numbers are given in various publications of the Bureau of 
Labor Statistics (Washington, D. C.). Current values appear semi-annually in a 
pamphlet, ‘Changes in Cost of Living,” which started July 15, 1935. Previous to that 
time (back to 1922) they were added twice a year (June and December) to the monthly 
pamphlet on retail prices. Previous to that, to 1917, they were compiled monthly. 
They are published at irregular intervals in the Monthly Labor Review. The basis for 
calculating cost-of-living index numbers was given in “Cost of Living in the U. $.” 
Bulletin of the U. S. Bureau of Labor Statistics, No. 357, May, 1924. Pp. 466. This 
method has recently been modified, as described in: Faith M. Williams, Margaret H. 
Hogg, and Ewan Clague, “Revision of Index of Cost of Goods Purchased by Wage Earners 
‘and Lower-Salaried Workers,” Monthly Labor Review, XLI (September, 1935), 819-37. 

221 The wholesale index numbers appear currently in a monthly pamphlet entitled, 
“Wholesale Prices.” Beginning in January, 1932, weekly index numbers were calculated. 
For earlier data, and method of calculating the index number, see “Wholesale Prices, 
1913-1928,” Bulletin of the U. S. Bureau of Labor Statistics, No. 493, August, 1929. 
Pp. 257. This is supplemented by Jater annual bulletins, No. 521 for 1929, No. 543 for 
1930, No. 572 for 1931. These wholesale index numbers also appear in the Monthly 
Labor Review. Bulletins No. 284 (1921), and No. 453 (1928), also deal with index 
numbers of wholesale prices; both are now out of print. 

222 George F. Warren and Frank A. Pearson, “A Monthly Index Number of Who’e- 
sale Prices in the United States for 135 Years,” Journal of the American Statistical Asso- 
ciation, XXVI (March, 1931 supplement), 244-9. 

223 George F. Warren and Frank A. Pearson, Prices, pp. 25-7. New York: John Wiley 


and Sons, Inc., 1933. Pp. 386. ‘ 
One may be interested in critical comments by Tucker, op. cit., (footnote 225). 
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the publishing of their reciprocals, which reflect the purchasing 
power of the dollar. Various authors, King,*** and Tucker,??° 
for example, have pointed out that such indexes do not prop- 
erly represent changes in the general price level. 

The series of index numbers of retail prices **° (which are 
practically confined to food, at present) go back to 1890. They 
are calculated monthly and are available separately for 
several food groups and for a number of cities. Data on the 
cost of coal, gas, and electricity are also secured and are 
usually published with the retail-price index of food. A new, 
general-purpose index of retail prices was projected by the 
Bureau of Labor Statistics during 1935-36, its appearance 
being scheduled for late in 1936. 

The National Industrial Conference Board has prepared its 
own series of index numbers on the cost of living,” which 
date from 1914. Differences between these and the index num- 
bers of the Bureau of Labor Statistics have been analyzed,?28 
Paul Douglas ° has critically examined some of the index 
numbers of the U. S. Bureau of Labor Statistics and has pre- 


224 Willford I. King, Index Numbers Elucidated, Chapter VIII, “Measuring the Pur- 
chasing Power of Money,” pp. 189-218. New York: Longmans, Green and Co., 1930. 
Pp. 226. 

225 Rufus S. Tucker, “Gold, Prices and Prosperity: No Statistical Basis for the 
Commodity Dollar,” The Annalist, XLIL (December 1, 1933), 707-9. è 

226 Index numbers for retail prices are given in a monthly pamphlet entitled, “Retail 
Prices.” Before March, 1932, this bore the title, “Prices: Wholesale and Retail.” The 
Monthly Labor Review also carries these index numbers. For a detailed recapitulation see 
the Bulletin of the United States Bureau of Labor Statistics, No. 495 (August, 1929). 
Pp. 219. This bulletin also describes the method of gathering and analyzing data fo 
the index numbers, pp. 11-3. 

Other descriptions of gathering data and calculating indexes are given in: “Methods 
of Procuring and Computing Statistical Information of the Bureau of Labor Statistics,” 
Bulletin of the U. S. Bureau of Labor Statistics, No. 326, 1923. Pp. 54. 

227 Basic figures and the method of calculating are given in: The Cost of Living in 

the United States, 1914-1930. New York: The National Industrial Conference Board, 
247 Park Avenue. 1931. Pp, 170. Recent figures are given in the supplement to the 
Conference Board Service Letter in March of each year. Other data appear in the 
monthly Conference Board Bulletin incident to articles. 
228 Elma B. Carr, “Cost of Living Statistics of the United States Bureau of Labor 
and the National Industrial Conference Board—Comparison of Methods Used in the 
Collection and Computation of Cost of Living Figures,” Journal of the American Statis- 
tical Association, XIX (December, 1924), 484-507. See also chapters in the preceding 
teference, and in the one next following. 

229 Paul H. Douglas, Real Wages in the U. S., 1890-1926. Publications of the Pol- 
lack Foundation for Economic Research, No. 9. Boston: Houghton Mifflin Co., 1930. 
Pp. 682. He discusses the construction of several sets of index numbers, presents some, 
and gives his own series of wholesale price numbers (p. 60), as well as others. 
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pared a revised set for the cost of living, together with index 
numbers for wages, and real wages. 

Index numbers of business activity are too numerous and 
varied to be reviewed. Mention will be made here of only a 
few, together with sources of many others.**° Under the direc- 
tion of Leonard P. Ayres, the Cleveland Trust Company has 
published two series known as “American Business Activity 
Since 1790,” (February, 1932), and “Business Activity and 
Four Price Series,” (March, 1932). These are maintained cur- 
rently by the Cleveland Trust Company’s monthly Business 
Bulletin. The index of business activity, like other similarly 
constructed series, represents a special form of index number, 
in which variations are taken not from a fixed base but from 
a variable estimated “normal” as the base. 

Six important business indexes are given by Leong,?"* for 
the years 1919-34, and other important series are referred to 
in his article. Whelden 2%? also gives an index of business 
activity for r919-34. The U. S. Bureau of Labor Statistics pub- 
lishes monthly index numbers of employment and payrolls.?** 
The survey of spendable money income, published periodically 
in Sales Management,*** gives an “index number” in addition 


230 Lists of current index numbers are given in the following references: 

Frederick E. Croxton and Dudley J. Cowden, Practical Business Statistics, pp. 397-404+ 
New York: Prentice-Hall, Inc., 1934. Pp. 529. See also the entirety of Chapter XVIII. 

Irving Fisher, The Making of Index Numbers, pp. 436-7. Boston: Houghton Mifflin 
Co., 3d ed., rev, 1931. Pp. 538. 

U. S. Bureau of Labor Statistics, Inder Numbers of Wholesale Prices in the United 
States und Foreign Countries, pp. 115-75. Bulletin No. 284, 1921. Pp. 350. 0. p. (Re- 
vision of Bulletin 173. 0. p.) 

231 Y., S. Leong, “Indexes of the Physical Volume Production of Producers’ Goods, 
Consumers’ Goods, Durable Goods, and Transient Goods,” Journal of the American 
Statistical Association, XXX (June, 1935), 361-76. An index of manufactures, and one 
for consumers’ goods excluding automobiles, are included in the article. 

232C. H. Whelden, Jr, “An Index of Business Activity—Manufacturing Values and 
Payrolls,” Harvard Business Review, XI (Summer, 1935), 503-12. 

233 “Employment and Payrolls,” monthly publication of the United States Bureau 
of Labor Statistics, Began in July, 1923, as “Employment in Selected Industries’; title 
has varied. 

Lewis E. Talbert and Alice Olenin, “Revised Indexes of Factory Employment and 
Pay Rolls, r919 to 1933,” Bulletin of the U. S. Bureau of Labor Statistics, No. 610, 
1935. Pp. 106. 

See also Real Wages, by Douglas, previously cited (footnote 229), especially p. 205, 
and Parts II-IV, including the appendices. 

284 Through 1930, contained in Section TI of the Annual Reference Number of Sales 
Management. Beginning in October 31, 1931 (Vol. XXIX) the “Survey of Spending 
Power? became a part of the regular numbers, and has appeared in April ro, 1933 
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to many other valuable population and commercial data for 
states, counties, and cities of 10,000 population. A number of 
important economic indexes are given in the publications of 
the National Bureau of Economic Research.*** 

Many indexes of current conditions are available in peri- 
odicals, especially the following: wegkly numbers of Tke 
Annalist, the monthly Federal Reserve Bulletin of the Federal 
Reserve Board at Washington, the Bulletin of the National 
City Bank of New York, the Monthly Review of the Federal 
Reserve Bank of New York, the Harvard Review of Economic 
Statistics, the Standard Statistical Bulletin, and the Survey of 
Current Business (published since 1921, by the U. S. Bureau 
of Foreign and Domestic Commerce). 

2. Educational applications. These various index numbers 
of economic conditions have been used in a variety of research 
studies in education, both as patterns of analysis and as direct 
sources of information. Mention has already been made that 
Phillips modified the Ayres’ Index Number of State School 
Systems; this was done by “deflating” the financial items ac- 
cording to variations in the index of the cost of living. The 
United States cost-of-living index numbers have also been used 
rather widely to account in part for the increasing costs of 
education since 1900; in fact, almost half of the increase in 
expenditures for education from 1900 to 1930 has been attrib- 
uted to the decreased purchasing power of the dollar during 
this period.” Another type of comparison is that between the 
increase in expenditures for education and the increases in 
(Vol. XXXII); by states only for April 20, 1934 (Vol. XXXIV); and by counties 
and cities again in April 10, 1935 (Vol. XXXVI). See corresponding numbers for later 
years. The 1931 number carried a cost-of-living index for each city in the U. S. over 
10,000 population, Address of publisher: Sales Management, Inc., 420 Lexington Ave., 
New York. 

235 See for example, Frederick C. Mills, Economic Tendencies in the United States: 

spects of Pre-War and Post-War Changes. No. 21 of the National Bureau Series. The 
‘vutional Bureau of Economic Research, 51 Madison Ave., N. Y., 1932. Pp. 639. Covers 
the thirty-year period, 1899-1929. Or see: W. I. King and ian Epstein, The National 
Income and Its Purchasing Power, 1930. Pp. 394. Current information is contained in 
the National Bureau of Economic Research Bulletin, published five times: a year. 

286 “Facts on School Costs,” Research Bulletin of the National Education Association, 
X (November, 1932), 223-4. A criticism of this conclusion has been given by Nelson B. 


Henry, “Index Numbers and the Cost of Schools,” American School Board Journal, 
LXIX (July, 1924), 58. 
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various indexes of industrial production and business activity. 
Index numbers for the cost of living %7 have been used to 
show variations in purchasing power of teachers’ salaries. *** 
Several cost-of-living index numbers have been constructed 
especially for teachers to reflect variations in costs from year 
to year or from place to place. For example, the report of the 
Committee on the Economic Status of the Teacher **° reviews 
the construction of special index numbers for the cost of living 
of teachers by four earlier writers and then constructs one of 
its own for the years 1928-34. Such studies are of basic 
importance in building salary schedules or in meeting the 
arguments of unfriendly groups. Harry ** made a study of 
differences in the cost of living of teachers in different parts 
of New York. This study was repeated in Ohio, and the index 
number was included as one of the factors recommended for 
use in the distribution of state school money.” Other studies 
may be found by consulting the topics, “Index Numbers,” and 
“Cost and Standard of Living,” in the Education Index. 
Preparation and use of index numbers. The objective char- 
acter of the data utilized in the index number, and of the re- 
sulting series of figures, should not obscure the fact that the 
components of the index number may not, individually or in 
the aggregate, provide an adequate representation of the char- 


237 The studies in the following volume may be interesting as illustrations of tech- 
nique in estimating costs of living: Standards of Living: A Compilation of Budgetary 
Studies. Bulletin No. 7 (revised edition). Washington: Bureau of Applied Economics, 
Inc, 1920. Pp. 156. 0. p. 

2a8 T. C. Holy, ‘“Cost-of-Living Indexes for Teachers’ Salaries,” Educational Research 
Bulletin, XII (February 8, 1933), 42-5- 

F. K. Shuttleworth, “Dollar and Real Incomes of Public School Teachers and of 
Wage Workers, 1889-1890 to 1934-35,” Educational Administration and Supervision, 
XXI (February, 1935), 81-96. 

See also Paul Douglas, footnote 229. 

239 The Teacher’s Economic Position,” Research Bulletin of the National Education 
Association, XIII (September, 1935), Chapter VII, pp. 222-42. See also circular No. 1, 
January, 1933, of the Educational Research Service of the National Education Associa- 
tion, on “Estimating Changes in Teachers’ Cost of Living.” 

240 David P. Harry, Cost of Living of Teachers in the State of New York. Teachers 
College Contributions to Education, No. 320. New York: Teachers College, Columbia 
University, 1928. Pp. 184. T 

241 ‘Equalizing Educational Opportunity in Ohio: A, Preliminary Report of a Survey 
of State and Local Support of Public Schools in Ohio, prepared under the direction of 
Paul R. Mort. The Ohio School Survey Commission, Columbus, Ohio, November 1, 1932. 


See pp. 39-40 and 147-9. 
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acter which one is seeking to index. The comments on “Under- 
lying Assumptions,” made in the preceding section, apply with 
equal force to index numbers. The Research Division of the 
National Education Association makes the following pertinent 
comments concerning the use of index numbers for rating 
school systems: *4? 


As purely objective measures of school efficiency which can com- 
mand universal acquiescence, none of the studies can be said to have 
succeeded. Statistical criteria for the measurement of state school effi- 
ciency possess the great advantage of definiteness, appeal to the public, 
and ease of recurrent application. But precisely because of these al- 
luring advantages certain dangers lurk in placing exclusive reliance on 
them and in hasty or uncritical interpretation of the comparative sta- 
tistics which quantitative criteria develop. Before it is possible to 
derive and to make proper use of quantitative criteria, the following 
difficulties must be faced and overcome: 

1. The data from which the criteria are computed must be com- 
parable. [That is, they must be uniformly defined and gathered from 
all states. This is at present far from the case.] 

2. Every quantitative criterion to be used must possess a demon- 
strable relationship to the efficiency of the schools. . . . It must be 
possible to show (either logically or statistically, and preferably both) 
that these factors are directly and intimately associated with school 
efficiency. Therefore, the selection of a given criterion . . . merely be- 
cause the data upon it are easily available, creates a second chance for 
error. 

3. The relative weight of different factors in affecting school effi- 
ciency must be known. . . . The practice of averaging a number of 
different factors to secure a single measure of school efficiency is based 
on the doubtful assumption that all factors considered are of equal 
importance. 

Results of such ratings should be cautiously interpreted, . . . 
[and] such ratings should be supplemented by such other estimates 
of school efficiency as are available. 


The foregoing requirements and other criteria can be 
summed up under the two general requirements for all re- 
search; namely, that the data must represent the concept (the 
problem) which one is studying, and they must represent the 


242 “Difficulties Involved in Statistical Analysis,” p, 109 in “Estimating State School 
Efficiency,” Research Bulletin of the National Education Association, X (May, 1932), 
No. 3. 
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conditions in the field. One has first to select factors (items) 
for his index number which will represent as faithfully and 
completely as possible the general characteristic which he 
desires to study. In the field of economics, selecting items 
which represent wholesale prices does not involve any diffi- 
culty, since anything sold at wholesale would answer the pur- 
pose. However, selecting items to represent more complex, 
specialized concepts, such as cost of living, general business 
conditions, or the efficiency of school systems, calls for a 
thoughtful analysis of the concept. 

Proper weighting is a second element in representing the 
~haracteristic one desires to study. In the field of price indexes, 
one may weight by the quantity sold, and there can be little 
argument.** In indexes for more complex characteristics, such as 
business activity or the efficiency of school systems, the problem 
becomes important, and generally it must rest on judgment 
after a careful analysis. There is a somewhat prevalent mis- 
conception that when one uses data as they are collected, the 
data are not weighted. They have, however, a natural weight- 
ing which is just as real (and may be just as wrong for a 
certain purpose) as any artificial weighting that might be as- 
signed. On the other hand, workers will frequently reduce all 
of the factors to equal weighting,“ usually with the thought 
that they have thereby relieved themselves of all responsibility 
for judgment and made their work perfectly objective. But 
equal weighting may be less justifiable than natural weighting 
or some other arbitrary weighting. The problem is primarily 
one of judgment, barring the extensive research which might 
remove judgment another step, and must be faced as such. 


243 A question may arise as to whether to weight by the quantities in the base or in 


the given year. Certain formulas use a ‘combination of the two, Exact weighting is not 
however regarded as important. See Irving Fisher, The Making of Index Numbers. Pub- 
lications of the Pollack Foundation for Economic Research, No. 1, pp- 328, 346-8, 
432, 447-9. (Third edition, revised.) Boston: Houghton Mifflin Co., 1927 (2d printing, 
December, 1931). Pp. 538- 

244 T. L. Kelley, Interpretation of Educational Measurements, pp. 66-74. Yonkers-on- 
Hudson, N. Y.: World Book Co., 1927. Pp. 363- 

Douglas E. Scates and F. R. Nofisinger, “Factors Which Determine the Effectiveness 
of Weighting,” Journal of Educational Research, XXIV (November, 1931), 280-5. 
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To have one’s data truly representative of the field condi- 
tions, the measures must be uniformly (and carefully) defined. 
For example, such an apparently simple thing as “one day of 
attendance” varies a great deal in its concept from one school 
system to another. Phillips gives an illuminating discussion of 
this difficulty.**° Again, the field must be properly sampled.2** 
That is, items or sources which are closely representative of 
other items or sources in the field must be gathered. In the 
case of the forty-eight states, all can be included, and the only 
sampling involved is that in the initial selection of items to 
represent the concept, and perhaps in the further definitions of 
these factors in field terms. In the case of prices, one cannot 
gather data on all of the prices of any single commodity in 
every city and village in the United States, so he resorts to 
sampling of the field and gathers data from a certain number 
of cities which he believes will also represent other cities.“ 

The problem of putting the data together and performing 
the statistical calculations, after the above aspects have been 
given due attention, is a technical one.2** The formulas for 
index numbers are well treated in statistical texts and will 
not be cited here. Fisher °*° gives the most elaborate treat- 
ment of formulas, though he does not exhaust the possibilities, 
He discusses six types of averages and six types of weighting 
(p. 351) and analyzes the resulting index numbers on the 


245 Frank M. Phillips, “Educational Rank of the States, 1930,” Section II, “Uniform 
Definitions, Records, and Reports,” American School Board Journal, LXXXIV (March 
and April, 1932). Also in a pamphlet of same title, published by the author, Wash- 
ington, 1932, pp. 25-40. 

246 King, op. cit. (footnote 224), p. 49, says that the problem of sampling the field 
is the only real problem of index numbers. Though other writers recognize the im- 
Portance of sampling, they do not concur in such an extreme emphasis. E.g., cf. Fisher, 
op. cit, (footnote 249), pp. 336-40, and 524-5. King’s position is developed in Chap- 
ter IV, “Sampling as Related to Index Numbers,” pp. s9-77, and Chapter VII, “Per- 
centages of Error Found in Certain Price Indices,” pp. 143-88 of his book, 

247 A description of the methods by which the United States Bureau of Labor Statistics 
gathers data from the field for its index numbers is given in the following bulletin: 
“Methods of Procuring and Computing Statistical Information of the Bureau of Labor 
Statistics.” Bulletin of the U. S. Bureau of Labor Statistics, No. 326. Washington, March, 
1923. Pp. 54. 

248 Tt is assumed that the student will familiarize himself with the references which 
have been cited, since a number of them describe the methods by which the indes 
numbers were prepared. See particularly, Mills, National Industrial Conference Board, 
United States Bureau of Labor Statistics, Douglas, and Whelden. 

249 Fisher, op. cit. (footnote 243). 
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basis of several criteria. He concludes that his formula No. 
353 is the “ideal” one (pp. 360 and 493), but that formula 
No. 2153 is more easily calculated and practically as good 
(pp. 361 and 494). His formula No. 53 (estimated to be cor- 
rect within 1 per cent, pp. 362 and 494) is both rapid and 
simple to explain to non-technical workers. It is the form used 
by the United States Bureau of Labor Statistics in calculating 
the index numbers of wholesale prices, retail prices, and cost 
of living. It will be found satisfactory for most purposes. His 
formula No. 1, which is a simple average, is the one that has 
been generally used in educational studies; Fisher says that **° 
«Tt should not be used under any circumstances, being always 
biased and usually freakish as well.” 

Other sources of index-number formulas are Kelley, 
Young,” and most books on statistical methods in economics. 
A practical and comprehensive treatment is given by Croxton 
and Cowden.?** One should also consult the Journal of the 
American Statistical Association for many practical and theo- 
retical articles. 

Most of the useful formulas are really not difficult, though 
at first they may appear to be. They are generally stated in 
terms which grow out of the economic field; for example, $ 
stands for the price of a commodity and q stands for the quan- 
tity of this item that was sold. To translate these symbols 
into non-financial terms, p would stand for the value observed 
for any particular item or character at any particular time or 
place, and q would be the weight that is assigned to this 


character. 
The use of index numbers has been reviewed here in some 


251 


250 Ibid., , 361, 466; see also pp. 64-6. 

251 ae L Kelley, Statistical Method, Chapter XIII, “Inder Numbers,” pp. 
331-47. New York: The Macmillan Co., 1923. Pp. 390. On pp. 344-5, Kelley’s formulas 
Servo ta, xg are respectively Fisher's formulas Nos. 2153, S$, and 353- ; 

252 Allyn A. Young, Textbook of ‘Mathematical Statistics, edited by H. L. Rietz, 
Chapter XII, ‘Index Numbers,” pp. 181-94. Boston: Houghton Mifflin Co., 1924. 
Pp. 221. Young’s formulas Nos. 10, 11, and 13 on p. 188 are respectively Fisher’s 


Ví , and 2153. a y i 
ee 53° 855 on and Dudley J. Cowden, Practical Business Statistics, Chapter 
XVII, “The Construction of Index Numbers,” pp. 362-77, and Chapter XVIII, “Some 
Sa ire sca” pp. 378-404. New York: Prentice-Hall, Inc., 1934, Pp. 529. 
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detail because nowhere else in educational literature is the 
subject treated as a unit. There is room for many more studies 
in education utilizing the index-number technique. 

School surveys. The final application of the normative-survey 
method to be discussed in this chapter is that of school sur- 
veys. Although surveys may be made for a number of different 
purposes, the discussion is introduced in connection with the 
topic of appraisal, since this is typically the essential element 
in school surveys. The survey is, however, a composite of 
normative-survey procedures, involving all of the various tech- 
niques which have been dealt with in these two chapters (VII 
and VIII) in order to realize its appraisal purpose. It can be 
well said that no other unified undertaking so fully represents 
the normative-survey method of research in all of its various 
phases as does the school survey. 

1. Development of the school survey. The survey of educa- 
tional practices in city school systems was patterned after the 
sociological survey at the outset, but rapidly developed char- 
acteristics of its own. There are a number of brief historical 
accounts of the development of surveys.’** The following para- 
graphs are quoted from Caswell, to give a sketch of what may 
be regarded as the modern survey movement.?°* 


The Boise, Idaho, survey of 1910 marks the beginning of the 
movement. In 1911 and 1912 the East Orange and Montclair, New 
Jersey, and the New York City surveys were made. By 1914 the idea 
had become so popular that surveys of nine city school systems were 
made by outside experts during the year. In that year also the Car- 
negie Corporation made, in Vermont, the first state-wide survey. Dr. 
Leonard P. Ayres, then with the Russell Sage Foundation, in 1916 
directed the most comprehensive survey of the Cleveland, Ohio, 
schools. This study was by far the most detailed of any made up to 
that time. 

254 Henry Lester Smith and Edgar Alvin O'Dell, Bibliography of School Surveys and 
of References on School Surveys, p. 179, I. “Early Investigations” (not all of these 
are to be regarded as surveys in the usual sense); p. 180, “History of Surveys” (14 
surveys). Bulletin of the School of Education, Indiana University, Vol. VIII (September 
and November, 1931), Nos. x and 2. Bloomington, Ind.: Bureau of Codperative Research, 
Pp. 212. (Two numbers published as one.) 

255 Hollis Leland Caswell, City School Surveys, pp. 5-6. Teachers College Contribu- 


tions to Education, No. 358. New York: Teachers College, Columbia University, 1929. 
Pp. 130. 
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During the years immediately following 1916 the results and the 
future of the movement were widely discussed. Many educational 
meetings throughout the nation gave prominent places on their pro- 
grams to such discussions. In this process of popularization such ex- 
travagant claims were made for surveys that the movement later 
came to be considered an educational fad by many leaders. In fact, 
many inferior surveys were made during this period by those who 
lacked the necessary training to direct them successfully. It is not 
strange that much bitterness was caused by certain of these surveys 
which were used for partisan rather than professional purposes. 

Following the World War, the popularity of locality surveys waned 
somewhat. Within the past few years, however, the number has 
gradually increased, until in 1927 more surveys of city school systems 
by outside experts were published than in any previous year. In addi- 
tion, three state-wide surveys were made. It now appears that locality 
surveys have passed the period of widespread popularization, weathered 
a time of depression, and are now increasing in favor. Fewer extrava- 
gant elaims are being made either in favor of or in opposition to 
surveys. 


2. Kinds of surveys. Surveys may be classified on any num- 
ber of different bases.2°* A few of these will be mentioned here. 
On the basis of area, surveys may be for cities, counties, states, 
regions, rural, urban, etc. Caswell refers to surveys covering a 
reasonably compact territory when he speaks of “locality sur- 
veys.” On the basis of level, there are surveys of elementary 
schools, junior high schools, colleges, universities, adult educa- 
tion, etc. Surveys are commonly classified by area and educa- 
tional enterprise, indicating the particular phase of the total 
educational problem with which they are concerned. For ex- 
ample, there are surveys of buildings, of reading, of vocational 
education, of administration, of finance, of pupil progress, etc. 
General surveys may be referred to as “comprehensive.” 

A fundamental basis of differentiation is whether surveys 
are educational, or sociological, or of some other general nature. 
Tt is assumed here that we are thinking of educational surveys. 
These may deal with public or private institutions. In either 
case the institutions maybe scattered over the country, such 
as in the case of land-grant colleges, and these cases are in 


256 See Smith and O'Dell, op, cit. (footnote 254). p. 170, IV, “Classes of Surveys.” 
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contrast to the “locality” survey. Another common basis of 
classification of surveys is that of agency. Surveys have been 
made by individuals, by organizations within university schools 
of education, by selected groups of college professors, by the 
United States Office of Education, by state departments of 
education, by the local staff, etc. 

3. Survey techniques. As stated above, school surveys utilize 
in one instance or another the whole range of normative-survey 
procedures. In one of his studies Caswell analyzed the tech- 
niques which were commonly used in surveys and tabulated 
their frequency of use by the various agencies that have made 
large numbers of surveys. He thus brought out the character- 
istic types of attack emphasized by different survey agencies. 
His list and description of techniques follow: *°* 


1. Analysis of available basic data. All cases in which data were 
secured from any regularly prepared record, report, or document were 
included under this heading; for example, the minutes of the Board of 
Education, the city charter, attendance reports, individual pupil 
record cards, etc. Usually these data have to be tabulated or treated in 
special ways; for example, an age-grade table is a special tabulation 
of data on the age and grade of pupils, which data are available from 
individual pupil record cards. Hence, the method of securing and treat- 
ing data for an age-grade study was classified as analysis of available 
basic data. 

2, Score card and rating scale. Any scheme whereby various ele- 
ments in a situation were listed and weighted or evaluated was in- 
cluded under this heading. Ordinary check-lists which had no standards 
or rating were included under observation, 

3. Standard tests. Both intelligence and achievement tests were in- 
cluded under this heading. Tests developed especially for use in a sur- 
vey and standardized as a part of the survey work were also included 
under this heading. 

4. Case study. The term case study is used here in a very general 
way. Whenever an individual pupil was studied and his case used as 
evidence to support a conclusion, this was classified as a case study. 

5. Experimental procedure. Whenever groups were equated with re- 
spect to certain characteristics and compared with respect to a variable 
characteristic, this was classified as experimental procedure. 

6. Interview or questionnaire. Whenever data were secured by 


257 Hollis Leland Caswell, “Survey Techniques,” Educational Administration and 
Supervision, XIX (September, 1933), 431-41. 
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means of conversation or by means of forms filled out by members 
of the staff of the school being surveyed, this was classified under the 
above heading. 

7. Observation. Whenever the basis upon which a problem was 
solved was a subjective examination, this was classified as observation. 


In studying fifty-one published surveys from 1914 to 1931, 
Caswell found that the experimental procedure was not used 
in any of them except one survey in 1929; case-study pro- 
cedures increased over the period from less than r per cent to 
5 per cent of all problems taken up; score-card and rating-scale 
techniques were employed for less than 1 per cent at first, and 
for ọ per cent of all problems taken up at the end of the period ; 
standard tests were employed in the solution of 2 per cent of 
the problems surveyed at the beginning of the period, and for 
6 per cent at the end; interviews and questionnaires decreased 
from 4 per cent to x per cent of the problems attacked, and 
observation decreased from 18 per cent to about half of that 
amount; the first technique, analysis of available basic data, 
continued throughout the period to furnish evidence for 70 
per cent to 80 per cent of all the problems attacked by the 
survey. 

The foregoing classification of school-survey techniques 
parallels closely that used in the present treatment of norma- 
tive-survey research; it includes case study and experimental 
procedure, which are treated in other chapters; it combines 
interview and questionnaire, which are regarded in the present 
text as distinct; and it omits special mention of documentary 
frequency studies. These latter studies would be included 
under the first technique. As an illustration of documentary 
frequency studies in surveys. one may note the study of board 
minutes to ascertain the frequency of mention of various 
topics—used as an index of the kind of problem to which 
the board devotes its attention.*** 

4. Literature on school surveys. Because survey reports in 
large numbers are readily available for direct study, and 


258 Leonard P. Ayres, School Organization and Administration, Chapters I-III. Cleve- 
land Education Survey. New York: Russell Sage Foundation, 1916. Pp. 135. 
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because the literature on surveys is well indexed, no elaborate 
analysis of surveys and their characteristics will be under- 
taken in the present chapter. One should pursue the subject 
in accordance with his interests, selecting those surveys which 
constitute examples of the type he desires to know. The 
_ following paragraphs will introduce the key publications in 
the literature on school surveys. 

The most comprehensive general treatment on surveys is 
a textbook prepared by Sears **° some years ago. This out- 
lines the nature of surveys and discusses the type of con- 
sideration which is appropriate to the various phases of the 
educational system being surveyed. More recent writings by 
this author will be found under his name in the Education 
Index. Perhaps special mention should be made of his “The 
School Survey as a Means of Training Field Workers.” *° 
This article should be read primarily for the concept of survey 
work which is expressed incidentally in connection with the 
various phases of the work discussed. The point of view 
which it represents should be clearly understood by every 
one who essays to make a school survey. 

The School of Education of Indiana University has pre- 
pared the chief indexes of this field. The Bibliography of 
School Surveys and of References on School Surveys *** lists 
surveys by locality, with national surveys at the end. In all, 
about 1800 surveys are listed; many of these are not the 
type ordinarily connoted by the term school survey, but their 
inclusion serves to make the bibliography more comprehensive. 
Part II of this work is a bibliography of references which 
deals with surveys—either particular ones, or the work in 
general. These are arranged by author. There follows a “Classi- 
fication of References on Surveys,” which affords a classified 
index to these articles, This index may be used by the student 
for the location of survey reports in which he is most inter- 
ested. If, on the other hand, one is interested to find out how 


259 Jesse B. Sears, The School Survey. Boston: Houghton Mifflin Co., 1925. Pp. 440. 
260 Jesse B. Sears. Journal of Educational Research, XXIX (October, 1935), 138-43. 
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various surveys have dealt with particular subjects or phases 
of the educational system and what they have recommended 
on them, he should consult the Topical Analysis of 234 
School Surveys.2* This publication is an index to the detailed 
treatments of the various survey reports which were included. 
It is a guide of utmost importance for material up to its 
date of publication (1927). 4 

One can find a discussion of the surveys made during each 
two-year period since 1916 by consulting the proper chapter 
of the Biennial Survey of Education in the United States, 
made by the U. 3. Office.of Education. These chapters are 
also printed as separate bulletins of the Office.** For current 
surveys, one should consult “Surveys,” and cross references, 
in the Education Index, and the topic “Surveys” or “Educa- 
tional Surveys” in the Index of the Bibliography of Research 
Studies in Education, published annually by the United States 
Office of Education. 

A review and appraisal of 230 surveys of higher institutions, 
prepared by Walter C. Eells, were announced for publication 
in 1936 by the Carnegie Foundation for the Advancement of 
Teaching, New York City. 

One should also learn something of the sociological sur- 
vey, both because it was the forerunner of the educational 


261 Topical Analysis of 234 School Surveys. Bulletin of the School of Education, 
Indiana University, Vol. III (March, 1927), No. 4. Bloomington, Ind.: Bureau of Co- 
operative Research. Pp. 111. 

262 The first of these was: E. F. Buchner, Educational Surveys. U. S. Bureau of Edu- 
cation Bulletin, 1918, No. 45. Advance sheets from the Biennial Survey of Education 
in the United States, 1916-1915. Washington, D. C., 1919. Pp. 56. 

263 Allen Eaton and S. M. Harrison, A Bibliography of Social Surveys. New York: 
Russell Sage Foundation, 1931. Pp. 468. 

Manuel C. Elmer, Technique of Social Surveys, (Third edition.) Los Angeles: Jesse 
Ray Miller, 1927. Pp. 260. s 
Paul V. Kellogg and Neva R. Deardorff, Social Research As Applied to Community 
Progress. International Conference of Social Work, First Section, New York; Russell Sage 

Foundation, 1928. 

George A. Lundberg, 
Social Survey,” pp. 140-67 and pp. 3. 
Institutions,” pp. 243-99, PP- 345-6, and pp. 361 
Co., 1929. Pp. 380. 6 

Robert S. Lynd and Helen Merrell Lynd, Middletown: A Study in Contemporary 
American Culture. New York: Harcourt, Brace and Co., 1929. Pp. 550. 

Dorothy C. Mott, “Community Survey Technic,” Department of Elementary School 
Principals Bulletin, XI (April, 1932), 203-11. 

Howard W. Odum and Katherine Jocher, An Introduction to Social Research, Chapter 


Social Research, Chapter VII, “Field Work: The Interview and 
36-8; Chapter X, “The Measurement of Social 
-70. New York: Longmans, Green and 
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survey, and because one may need to gather similar informa- 
tion as a part of an educational survey, or for direct use in 
instruction or administration. 

5. National surveys. Because of their general interest atten- 
tion is called to the national surveys undertaken by the United 
„States Office of Education. Mention will be made first of 
the Biennial Surveys of Education, referred to in a preceding 
paragraph. These consist of data reported to the United States 
Office of Education on regular report forms every two years 
(both states and local districts make these reports), and of 
interpretations or discussions of these data. These Biennial 
Surveys began in 1916-18.*% 

Some idea of the scope of these reports may be obtained 
from the following list of topics treated. In Voiume I of the 
report for 1928-30,’ there were twenty-three chapters, cov- 
ering, among other subjects, elementary education, secondary 
education, industrial education, commercial education, home- 
making education, agricultural education, art education, music 
education, hygiene and physical education, education of ex- 
ceptional children, adult education, college and university 
education, education of teachers, education of certain racial 
groups, radio and education, library service, and educational 
legislation. In Volume II, which is almost wholly tables of 
statistics, there were seven chapters containing statistics of 
state school systems, of city school systems, of universities, 
colleges, and professional schools, of high schools, of private 
schools, of expenditures, etc. The data are extensive. These 
Biennial Surveys constitute an important source of informa- 
tion on current conditions and trends in education in the 
XVI, “Types of Method: The Survey,” pp. 244-59. New York: Henry Holt and Co., 
1929. Pp, 488. 

‘Thomas E. Sullenger, “Approaches to Methods of Community Study,” Sociology and 
Social Research, XVIII (May, 1934), 470-6. Excerpts in Journal of Educational Sociol- 
ogy, VIII (October, 1934), 94-7- 

See also the field studies cited in connection with the interview, footnotes 31-6, See 
also the topic, “Social Surveys,” in the Education Index. 

264 Biennial Survey oj Education. United States Bureau of Education Bulletin, 1919, 
No. 91. Washington, 1920. Pp. 797. 


265 Biennial Survey of Education. United States Office of Education Bulletin, 1931, 
No. 20, Washington, 1932. Vol. 7, pp. 807; Vol. 2, pp. 833. 
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United States. Survey reports following the one for 1928-30 
have been curtailed, owing to shortage of funds. 

During the past decade, the United States Office of Educa- 
tion has undertaken five special surveys on a national scale, 
in addition to its regular biennial surveys of education in 
general. These five surveys include: the Survey of Land-Grant 
Colleges and Universities,*°° begun in 1926; the Survey of 
Negro Colleges and Universities,” begun in 1927; the Na- 
tional Survey of Secondary Education,*"* begun in 1929; the 
National Survey of the Education of Teachers,* begun in 
1930; and the National Survey of School Finance," begun in 
1931. These national surveys of special phases of education 
are described in one of the government bulletins." 

5. Building surveys. A second set of surveys which will be 
given special mention, because of their general interest, are 
surveys of school buildings”? These generally constitute a 


266 Survey of Land-Grant Colleges and Universities, Edited by Arthur Klein, United 
s Office of Education Bulletin, 1930, No. 9. Washington, 1930. 2 Vols., pp. 998 


I. 

267 Survey of Negro Colleges and Universities, Arthur J. Klein. United States Office of 
Education Bulletin, 1928, No. 7. Washington, 1929. Pp. 964. 

National Survey of Secondary Education. United States Office of Education Bul- 
letin, 1932, No. 17, Monographs Nos. 1-28. Washington, 1933-1935. 28 Vols. 

See “National Survey of Secondary Education” in the Education Index for 1929-1932 
and for 1932-1935, for ninety-five references relating to this survey. 

269 National Survey of Education of Teachers. United States Office of Education Bul- 
letin, 1933, No. 10, Volumes I I. Washington, 1933-1935. 6 Vols. 

See “National Survey of Education of Teachers” in the Education Index for 1929- 
1932 and for 1932-1935 for thirty-three references relating to this survey. 

270 National Survey of School Finance. Owing to the failure of continued appropria- 
tions, this survey was stopped at the end of its first year. One volume of the report was 
published by the Office of Education and two volumes by the American Council on 
Education. 

Carter Alexander and Timon Covert, Bibliography on Educational Finance, 1923-1931. 
United States Office of Education Bulletin, 1952, No. 15. Washington, 1932. Pp. 344. 

Research Problems in School Finance. Washington: American Council on Education, 
1933. Pp. 164. 

Paul R. Mort and Others, 
School Finance, Publication No. 3, 
Finance,” in the Education Index for 1929-1932 and for 1932-1935, 


4 
State Support for Public Education. National Survey of 
1933. Pp. 496. See “National Survey of School 
for 30 references 


“National Surveys of the Office of Education,” Chapter XX, in 
ited States, 1928-1930. United States Office of 
Educati tin, , No. 20. Washington, 1931. Pp. 32. 
esate ae Aa ee Henry Lester Smith and Edgar Alvin O'Dell, 
Bibliography of School Surveys and oj References on School Surveys, those surveys 
marked. “bld” after “Type.” For references about building surveys, see p. 179. (Com- 
plete reference is given in chapter bibliography.) /) 

The building surveys in the above publication, about 230 of them, are listed, by num- 
ber. in item No. 2442, in Henry Lester Smith and Forest Ruby Noffsinger, Bibliography 
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preliminary step in the formulation of a building program ** 
which is to cover a period of years. They are frequently 
included as a part of a general or comprehensive school survey. 
They normally embrace an appraisal of the existing school 
buildings?" as a basis for determining what buildings should 
be rehabilitated or replaced at once and for estimating when 
other buildings will need to have additions, extensive repairs, 
or replacements. Such an appraisal makes use of the evaluat- 
ing procedures described earlier. 

A second important element of the building survey is the 
estimation of present and future needs. This step involves a 
study of the present utilization of buildings ** to ascertain 


of School Building, Grounds, and Equipment, Part II. (For complete reference, sce 
chapter bibliography.) 

Tn addition to the foregoing, many of the surveys marked “comp” in Smith and 
O'Dell contain surveys of buildings as a part of the comprehensive survey. These can be 
identified only by inspecting the survey reports. 

For current surveys, consult the Education Index; ‘“Surveys—School Buildings.” 

For a bibliography on building programs, including surveys incident to them, see 
“Preliminary Considerations and the Development of the Building Program,” in Henry 
Lester Smith, and (Leo M. Chamberlain or Forest R. Nofisinger), A Bibliography of 
School Buildings, Grounds, and Equipment: Part I, pp. 48-53, and Part IV, pp. 28-32. 
(See chapter bibliography for complete reference.) 

A comprehensive review is given by Harry P. Smith, “The Survey Preliminary to a 
School Building Program, Chapter I, pp. 346-63, in “School Buildings, Grounds, Equip- 
ment, Apparatus, and Supplies,” Keview of Educational Rescarck, 11 (December, 1932), 
No. 5. 

Fred Engelhardt and N. L. Engelhardt, Planning School Building Programs. New York: 
Bureau of Fublications, Teachers College, Columbia University, 1930. Pp. 574. 

Arthur B. Moehlman, Public School Plant Program. Chicago: Rand McNally and Co., 
1929. Pp. 405. 

For a book presenting school-building programs in ten cities, growing out of surveys, 
see: N. L, Engelhardt, School Building Programs in American Cities. New York: 
Teachers College, Columbia University, 1928. Pp. 560. 

The Planning and Construction of School Buildings, Thirty-Third Yearbook of the 
National Society for the Study of Education, Part I. Bloomington, Ill.: Public School 
Publishing Co., 1934. Pp. 337- 

Russell A. Holy, The Relationship of City Planning to School Plant Planning. Teach- 
ers College Contributions to Education, No. 662, New York: Teachers College, Columbia 
University, 1933. Pp. 135. 

For a bibliography on the building program of higher educational institutions, see sec- 
tion IV of Henry Lester Smith and Forest Ruby Noffsinger, Bibliography of College and 
University Buildings, Grounds, and Equipment. Bulletin of the School of Education, 
Indiana University, X (March, 1934), No. 2. Bloomington, Ind.: Bureau of Co- 
operative Research. Pp. 199. 

See ‘School Building Programs,” in the Education Index. 

274 See the following sections in the Review of Educational Research: II (December, 
1932), 354-7; IV (February, 1934), 52-3. 

See the section, “Score Cards and Building Measurement,” in Bibliography of School 
Buildings, Grounds, and Equipment (footnote 273), Parts I, II, and IV. 

275 See the following sections in the Review of Educational Research: IL (December, 
1932), 357-8; IV (February, 1934), 53-4; V (October, 1935), 340. 

References to utilization are given in the section, “Score Cards and Building Measure- 
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how completely the existing plant is filled, and it involves 
a prognostication as to the general population growth of the 
city, as well as an estimation of the growth in school popu- 
lation.” Further, an estimation of the direction in which 
cities will extend, and of the growth in population density 
in various parts of the city, is necessary in order that specific 
recommendations may be made as to location of new build- 
ings. The recommendations must, of course, be made with 
consideration of the present plant, the ultimate plant that is 
thought to be desired,?"* and the financial program that will 
be required.” 

7. The school survey and educational science. Although a 
survey should be made for the benefit of educational practice 
in the field, it would be wrong to assume that this is its only 
contribution. Educational science, as represented primarily by 
the viewpoints of educational professors, is in part dependent 


ment,” of the Bibliography of School Buildings, Grounds, and Equipment (footnote 273), 
Parts I, II, and IV. 

276 For references on prediction of population growth, see the following sections in the 
Review of Educational Research: TI (December, 1932), 348-545 IV (February, 1934), 
49-52. 

See the section, “Studies in Population and Pupil Accommodation,” in Parts I, II, and 
IV of the Bibliography of School Buildings, Grounds and Equipment, cited in footnote 
273. 

Arthur B. Moehlman, op. cit. (footnote 273), pp. 52-90, 299-300, and 323-83. 

N. L. Engelhardt and Fred Engelhardt, op. cit. (footnote 273), Chaps. I-IV, pp. 1-90. 
A critical study of methods of forecasting population growth has been made by: Leo 
Martin Chamberlain and A. B. Crawford, Prediction of Population and School Enrollment 
in the School Survey. Bureau of School Service Bulletin, IV (1932), No. 3. Lexington, 
Ky.: University of Kentucky. Pp. 27. 

277 Review of Educational Research, II (December, 1932), 359-61; IV (February, 
1934), 54-5; V (October, 1935), 337-43- 

In the Bibliography of School Buildings, Grounds, and Equipment, see the section, 
“Adaptation of the Building to Specific Educational Needs,” in Parts I, II, and IV. 
(See footnote 273.) 

See the topic, “School Buildings and Equipment,” in the Subject Index of the annual 
Bibliography of Research Studies in Education, issued by the U. S. Office of Education, 
for pertinent studies. 

See the topic, “School Buildings—Designs and Plans,” in the Education Index. 

278 For references on financing the school-building program, see the Review of Educa- 
tional Research, as follows: II (December, 1932), 361-3; IV (February, 1934), 55-7. 

In the Bibliography of School Buildings, Grounds, and Equipment (see footnote 273); 
see the section on “Building Costs” and the three following sections, in each of Parts 
I, II, and IV. 

Fred Engelhardt and N. L. Engelhardt, op. cit. (footnote 273), Chapters XVI-XVIII, 
pp. 388-482. 

The Planning and Construction of School Buildings, op. cit. (footnote 273), Chapters 
XXXII-XXXIV, pp. 309-33. 

See “School Buildings—Cost,” in the Education Index. 
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upon the improvement of practices in the field. The latter 
are, of course, also dependent in part upon the insights and 
perspectives which the professorial group hold. The two are 
interrelated. It is only natural, therefore, that surveys which 
bring the best of educational thought (whether from the 
field or from universities) to bear upon practical problems 
will make a significant contribution to educational science ; 
this, in turn, will have an influence upon field practices by 
being inculcated in the thinking of future teachers and admin- 
istrators in training. 

Caswell has emphasized this point of view. After discussing 
the purposes for which school surveys have been made and 
their observable effects upon the school systems, he states: °°° 


The importance of the movement may be by no means completely 
described by an analysis of . .. direct local effects. Educational 
progress owes a larger debt to the survey movement for its influence 
on the development of scientific standards and objective methods of 
measurement for use in the administration of schools. The survey has 
served to relate the work of the educational expert in schools of educa- 
tion and state departments to individual local school systems. The 
problems of the field have been brought to the professor of education, 
and educational theory and methods have been carried to the field. 
Through this interaction a considerable body of scientific administra- 
tive standards and methods of measurement has developed. 


Undertaking a school survey. Not infrequently students 
undertake surveys of small cities or counties as Masters’ 
theses.**° If one does this, he should recognize that such a 
problem is likely to be larger than the average Master’s 
thesis problem, and that he needs to be in a strategic position 
to carry it through. That is, he should be a superintendent, 
or a principal who is well known and who has the complete 
approbation and interest of the superintendent in his project. 
The experience of conducting a minor school survey should 
afford desirable training in the gathering of field data. It 


279 Caswell, op. cit. (footnote 255), p. 81. 

280 For examples, see the topic, “Surveys,” in the annual Bibliography of Research 
Studies in Education, issued by the U. S. Office of Education. Theses can ordinarily be 
borrowed from university libraries by one’s local school or public library, for inspection. 
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must, however, be recognized that any survey conducted by 
an individual student is likely to deal with only a limited 
number of aspects of the school system and will probably 
be relatively mechanical as regards discovering real difficulties 
or their fundamental causes. It would seem better for an 
institution to organize a group of students to undertake a 
certain school survey, with the staff members giving consid- 
erable time to supervising the work, in an effort to see that 
it is properly codrdinated and directed to underlying problems. 
The matter of assigning thesis credit for such a group project 
should not prove an insurmountable problem. 

The objective steps. Certain of the more obvious steps in 
carrying through a survey will be mentioned here. These are 
practically self-evident, but will be reviewed by way of 
systematization. 

1. Preparation of plans. In undertaking a survey, one will 
make as careful plans as possible in advance, concerning the 
scope of his survey, the intensity of his study, the amount 
of time which should be devoted to each phase, when each | 
step should be completed in order that he may finish in proper 
time, and the amount of money each step will require. Such 
general plans will be modified as the work progresses, but 
one must at least start out with a workable plan, and later 
modifications must be kept within the limits of possibilities. 

2. Preparation of schedules. Data-gathering instruments 
have been discussed throughout the two chapters on the 
normative-survey method, both as to use and construction. 
One will need them in a survey. The extent to which schedules 
of various kinds are used will depend largely upon the kind of 
survey one is making. If one is making a survey of build- 
ings, he will certainly use score cards of some kind, together 
with report forms on various aspects of utilization, etc. If 
one is studying pupil achievement, tests will be called for. 
Tf one is making a survey of schools over a large area, such 
as the survey of all the institutions of a certain kind in a 
state, he will undoubtedly have to rely upon reports secured 
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through specially prepared blanks.: In a city school survey, 
one may desire to have teachers answer a questionnaire; on 
the other hand, needed data may be available in the super- 
intendent’s office. 

The Division of Field Studies of the Institute of Educa- 
tional Research, of Teachers College, Columbia University, 
has prepared a systematic schedule of aspects of the school 
system to look for when its staff of workers goes into the 
field.2*? This schedule fills over one hundred pages. In similar 
fashion, but in different form, Cocking has prepared a check- 
ing list™ to be used by workers in making a school survey. 
This extends over several hundred pages. 

3. Gathering data. This step represents the application or 
use of the schedules, or other data-gathering procedures, in 
the field. The reader is referred to the discussion of survey 
techniques, given earlier.*** One should be prepared to dis- 
cover many things from interviews, some of which he can 
record in his findings, and some of which he must regard 
as confidential. In this connection, tact and strict ethics are 
important. 

4. Interpretation of data. This step is the same in all 
research. What do the findings really signify? One must ex- 
amine carefully the contributing factors; he must go out into 
the field and observe; he must know more than simply what 
his data show, if he is to interpret them correctly. Of course, 


281 See for example, Report of a Survey of the State Institutions of Higher Learning in 
Indiana, Charles H. Judd, Chairman. Indianapolis: State Board of Education, 1926. 
Pp. 124. 

282 Classification of Procedures to Be Followed in the Development of a School Survey. 
Prepared by the Division of Field Studies, Institute of Educational Research, Teachers 
College, Columbia University, September, 1931. Pp. 135. Mimeographed. 

285 Walter D, Cocking, Checking List for a School Survey. Nashville, Tenn.: George 
Peabody College for Teachers. Mimeographed. 

284 Other techniques adapted to special purposes may be suggestive: 

“A Self-Survey Plan for State School Systems,” Parts I and II, Research Bulletin of 
the National Educational Association, VIII (1930), 53-88, 89-164. 

F. W. Cozens, “The Technique of a School Survey in Health and Physical Education,” 
Research Quarterly of the American Physical Educational Association, III (December, 
1931), 3-20. 

Walter B. Jones, “Self-Survey in Urban Universities,” Educational Record, X (January, 
1929), 51-8. 

Earl L. Terman, “The Development and Application of National Educational Survey 
Techniques.” Doctor's thesis. New York: New York University, 1920 
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he must not read into his data things that are not there, 
but his interpretation must be made in the light of some 
understanding of the situation as it is. 

5. Preparing the report. If the survey is made as a thesis, 
the report may follow along the usual thesis lines. If it is 
made for the school system, the report must be carefully 
prepared for a large and varied group of readers. One’s tech- 
nical details will fail to impress board members and the 
public; they will in fact tend to make such persons disregard 
the survey as impractical or academic. In writing such a 
report, one will consider the point of view of the person 
who is not trained in statistics or in education, but who 
nevertheless has strong interests and convictions in and about 
education. Write the report so that it will be convincing to 
such a reader. 

6. Follow-up service. If the survey is made for the benefit 
of the school system and for the purpose of bringing about 
actual improvements where they appear to be needed, the 
survey staff should plan to keep in touch with the school 
system in order that stimulation and counsel may be available 
for carrying out some of the recommendations. In such a 
relationship, the services of the surveyors must of course be 
offered and not thrust upon the school system; after all, the 
operation of the local system is the responsibility of the 
local group and not of the survey staff. If one makes a 
survey as a principal working in the system, he probably should 
refrain from making comments after the survey is finished, 
leaving the practical application to the superintendent. 

A survey as field research. In calling attention to the above 
steps which are relatively routine, there was no intention 
of suggesting that a survey should be a mechanical application 
of steps and techniques. Rather, the survey is to be regarded 
as a research problem worked out in the field. As such, one 
will proceed with the usual attitudes of discovery and suc- 
cessive refinement of purposes. One should avoid the frame 
of mind of descending upon a school system with his sched- 
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ules all made out, securing authoritative codperation in get- 
ting them filled out, and then retreating to determine what 
he shall say about the schools. One must have some plans 
in advance, but at the same time he will expect to locate 
many of the pressing problems through interviews in the 
field. As in other forms of research, one will center attention 
on a problem, formulate a working hypothesis, gather data 
which will throw light on this hypothesis, interpret the data 
and evaluate his findings, and perhaps start over again to 
seek other data that will shed more light on the same or on 
a new hypothesis. In field work one cannot so readily repeat 
and follow up, since time generally does not permit ; he should 
therefore spend sufficient time in the field at the beginning 
in order to secure orientation, in the hope of avoiding un- 
profitable studies. 
Sears makes the following comments: **° 


A survey is not an inventory and prescription, but a diagnosis and 
prescription. It does not focus primarily upon the schools, or upon the 
data, but, through an analysis of both, seeks to discover the real prob- 
lems. These problems are almost invariably found to occur, not in 
isolation, but in combination. The identification of the fundamental 
problems is a major aspect of the survey attack. 


The underlying program of a survey. The technique and steps 
of a survey have been discussed. The importance of adapting 
procedures to an increasing insight into field conditions has 
been pointed out. It seems appropriate at this point to discuss 
some of the fundamental considerations of the survey under- 
taking which will keep the staff oriented in carrying through 
its numerous activities. 

Although the particular things that are done will vary with 
the locality and the kind of survey being made, there is a 
larger pattern of purposes which are probably appropriate for 
every survey. These are presented in the following paragraphs. 

1. What is the purpose of the survey? Although the purpose 


285 Jesse B. Sears, in 2 personal letter to the authors. 
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will usually be implied or stated in rather general terms when 
an agency is requested to make a school survey, this purpose 
will need to be refined and crystallized before the survey is 
undertaken. This Step is similar to that of working over a very 
general statement of a research problem until it becomes defi- 
nite in its essential details and can be used as a guide to the 
worker. The survey staff will desire to know the specific prob- 
lems which it is to attack, and whether it is to confine its 
attention to these or is to study related problems as they 
emerge. Even if the staff is advised that the survey is to be a 
comprehensive one, embracing all aspects of the school system, 
the survey staff will proceed cautiously at first until the true 
nature of the problems existing in the field begins to be appar- 
ent, and the purpose of the survey begins to assume more 
definite form. 

2. What are the educational needs of the community? If a 
survey is to be fundamentally sound, its recommendations 
must be based not only on a sound general philosophy of edu- 
cation, but upon a detailed examination of the needs of a 
particular community. (If the survey is not a “locality” sur- 
vey, the purposes of the institutions being surveyed may be 
substituted for the needs of the community, although even in 
such cases a question may be raised as to whether special in- 
stitutions do not need to be considered in the light of other 
factors in the locality.) One cannot adequately appreciate the 
educational problems of a school system or properly understand 
the objective findings, until he has secured a comprehension of 
the total social situation existing in the locality. He must be- 
come acquainted with the social ideals of the people, their 
traditions, their habits of expressing themselves in their daily 
lives, and give these attention, along with his consideration of 
the larger objectives and social ideas of the people of the 
nation, The surveyor must envisage the whole cultural life of 
the people, including their various institutions and folkways, 
and study the actual and potential service of public education 
in this setting—without losing sight of the fact that this com- 
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munity is a part of the nation, and that education must serve 
common state and national purposes also. 

To assume that public education is and should be the same 
throughout a state, or throughout the nation, is to imply that 
local groups have no right to a degree of individuality, by 
which they make unique contributions to the interesting and 
colorful pattern of national life, and it is to place a severe 
handicap on the service which education can render to the 
community in its many and varied aspects of life. One must 
see the educational needs of a nation and of a state; but he 
sees these only partially if he neglects the educational needs 
of the community, for the various communities, with their 
differences in social values, are the state and nation in a 
democracy. 

3. How well are present practices meeting the needs? This 
phase of survey work is the evaluative one. Having compre- 
hended the opportunity for education in this locality, and hav- 
ing gathered evidence of the actual service which the schools 
are rendering, the surveyor compares the two pictures. One 
will of course consider trends and the probable future situ- 
ation, as well as the conditions at the instant. How well a 
survey staff can make this comparison depends on how thor- 
oughly it has secured information on the needs and on the 
practices, and also on the ability of the surveyors to interpret 
these facts and to make distinctions which are of basic social 
significance. One needs a general understanding of human 
nature and a perspective which includes the standards and 
urges of all elements of the population, in order to see and to 
express properly the deviations of the current educational prac- 
tices from the apparent opportunities. Let not one who has only 
a technical or theoretical education undertake such a task! 

4. How efficient are present educational procedures? This 
question is clearly much narrower than the preceding one. 
Tt is concerned, not with what education should be doing, but 
with the detailed procedures of what it is doing. In other words, 
for the given expenditure of money and energy, are the returns 
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as large as they should be? This is the type of question which 
receives most consideration in surveys of the usual kind. In 
answering it, the surveyors will, of course, draw upon their 
own judgment, but they will lean most heavily upon compar- 
isons of the normative-survey sort. That is, what are other 
communities putting forth in the way of effort, and for this 
amount of effort, what are the returns? The data for the local 
schools are compared with these other data showing conditions 
in other places. In making such comparisons, all of the cautions 
concerning gathering and interpreting data, which are men- 
tioned at other places in this chapter and throughout this book, 
should be borne in mind. The comparison, even of objective 
facts, requires circumspection and judgment. A great variety 
of factors enters into every objective fact which must be con- 
sidered whenever that fact is used. 

5. What constructive measures can be undertaken to improve 
the contribution which the schools are making to life in that 
community? This, the final phase of the survey, is the most 
crucial. It utilizes the data and conclusions from all of the pre- 
ceding phases of the work and sets these in a plan of progress 
which recognizes both the difficulties of making changes and 
the desirability of achieving certain new results. It calls prob- 
ably for more insight than any of the preceding steps, for it 
must include them and interpret them in terms of a. working 
program. It is human nature to resist change and to resent the 
implication of any shortcomings. The recommendations which 
are made must be presented in such a manner as not to end in 
a defensive reaction on the part of the local school group, but 
to stimulate a desire to achieve according to the larger oppor- 
tunity which is pictured for them by the survey. 

The appraisal of educational service. The problem of evalu- 
ating educational institutions has given rise to an important 
question of whether to rate the objective aspects of an in- 
stitution, including some of its administrative provisions and 
practices, or whether to measure the educational attainments 
of its pupils. The two approaches are sometimes referred to as 
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securing static versus functional data, or as accounting versus 
measuring, or as concern over conditions versus outcomes. Each 
of the two methods has its advocates. 

The measurement of pupils’ educational outcome would seem 
to be the more rigorous approach. Apparently one is here deal- 
ing directly with the very purpose for which the institution 
exists, and any other approach is merely a makeshift. It should 
be pointed out, however, that few educational institutions 
would be willing to agree that they exist solely for the de- 
velopment of the knowledge and skills that are covered by 
attainment tests. The hopes and aspirations of presidents, 
deans, principals, and faculties are generally couched in much 
more inclusive terms. Available tests measure only a part of 
what these people are striving to develop and, according to 
many of them, not the most important part. 

Tt must be said on the other hand that it is not fair to credit 
or charge the institution entirely with the responsibility for 
having developed the attainment of its pupils to the level that 
may be observed. Pupils differ widely when they enter an in- 
stitution, and in ways which the institution usually cannot 
entirely eradicate. Many private institutions effectually select 
students who already possess in large degree the characteristics 
for which they desire to have the institution known. Institu- 
tions which do not control the extracurricular life of students 
cannot contribute as large or as certain an influence as those 
where students live under their regulations. It must be recog- 
nized that the character of the student’s leisure-time activities 
may exert as strong an influence on his developments as do the 
hours of fotmal study and training. It is not within the limits 
of common reason to take general measures of attainment as 
final indices of the value of institutions. 

Neither, on the other hand, does it seem wholly satisfactory 
to use measures of the more objective elements of an institution 
as the sole bases for determining its worth. There are many 
devices, services, and various provisions which are established 
with the hope that they will contribute to the desired develop- 
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ment of the students. Undoubtedly many do, but many fail 
because they are carried out mechanically or without proper 
perspective or interest. Many procedures, also, are established 
for the convenience and benefit of the administration, rather 
than directly for the welfare of the student group. Tt should 
be clear that the conditioning factors which an institution seeks 
to provide are not in themseives a sufficient guarantee of their 
own effectiveness. 

As a matter of general logic, it would seem desirable to in- 
clude evaluations of all of the elements in the situation which 
are significant indicators of good educational influence. One 
seeking a thorough rating of an institution should use tests of 
many different kinds for measuring certain attainments of the 
pupils; he should use check-lists, rating scales, or score cards 
for appraising many of the service provisions of the institution ; 
and probably in addition to these two, he should seek other 
lines of evidence, such as reputation, subsequent success of stu- 
dents, the judgment of those who are used to inspecting and 
rating institutions, the insight of those who are “on the inside” 
with reference to what is actually going on in the institution, 
etc. All of the facts which he can gather should be regarded as 
throwing light on various aspects of the service of the institu- 
tion and as playing some part in determining his final appraisal 
of the institution. 

Attention may be directed to a number of procedures which 
are advocated or actually used in appraising educationat 
service.**° Check-lists have been prepared by Wade, the Na- 
tional Education Association, and Frederick and Schorling, for 
school systems or institutions in terms of their provisions and 
practices. A score card for the same purpose has been prepared 
by Mort and Hilleboe. The North Central Association **7 has 


286 References given earlier in this chapter will not be repeated here. They may be 
located readily in the footnotes. 

287 The standards of the North Central Association are adopted each year, usually 
without significant alteration. See, for example, the “Policies, Regulations, Standards, 
and Recommendations for Accrediting Secondary Schools,” in the “Proceedings of the 
Commission on Secondary Schools,” North Central Association Quarterly, X (July, 1935), 
g6-1or, and “Standards for ‘Accredited Institutions of Higher Education,” in “Proceec 
ings of the Commission on Institutions of Higher Education,” North Central Association 
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generally used the method of appraising provisions, including 
such items as the pupil-teacher ratio, the library facilities, the 
endowment of current income per pupil, etc. Lundberg *** lists 
a number of schedules for appraising social institutions and 
practices, which are based primarily upon objective elements 
that presumably contribute to service and tend to affect its 
quality. 

Trimble **° discusses both types of measurement, but defi- 
nitely favors tests of attainment. H. P. Smith ?® used achieve- 
ment tests, supplemented by ratings on work habits, attitudes, 
and personality, in studying the value of the junior high-school 
organization. In discussing guidance, Kefauver **' reviews 
measures of various kinds which have been applied and con- 
cludes that a rigorous experiment, with measured results of 
guidance, is necessary before we know anything regarding its 
value, Such an experiment is essential, he points out, as a means 
of validating the other instruments or procedures which are 
commonly used. Such a contention may be reviewed by the 
reader in the light of the preceding discussion. 

The score cards, rating scales, and index-number techniques 
all place reliance upon the conditioning factors, except as they 
may be applied to classroom activities of pupils. The new 
criteria, worked out by the North Central Association and by 
associated accrediting groups for the rating of secondary 


Quarterly, VIII (June, 1933), 81-5. These standards are also published separately by the 
Association. 

Standards for other years may be found by consulting the Index of the North Central 
Association Quarterly, or by referring in the Education Index to North Central Associa- 
tion of Colleges and Secondary Schools, sub-topic: The Commission on Secondary Schools, 
or, The Commission on Institutions of Higher Education. 

For revised standards, see footnotes 292 and 293. 

288 George A. Lundberg, op. cit. (footnote 263). 

289 Otis C. Trimble, “Comparisons of Two Methods of Measuring the High School,” 
pp. 35-46 in Twentieth Annual Conference on Educational Measurements. Bulletin of the 
School of Education, Indiana University, X (December, 1933), No. 1. Bloomington, 
Ind.: Bureau of Codperative Research. Pp. 98. 

290 Harry P. Smith, “The Relative Efficiency of the Junior High School vs. the Con 
ventional 8-Grade Type of School,” Journal of Educational Research, XXIX (December, 
1935), 276-80. 

291 Grayson N. Kefauver, “Proposals for a Program of Evaiuation of Guidance,” 
School Review, XLII (September, 1934), 519-26. See also G. N. Kefauver and H. C., 
Head “Evaluation of Guidance Programs,” Occupations, XII (March, 1934), 106-14, 

art 2. 
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schools 2°? and higher institutions, continue to emphasize 
practices and provisions, with some attention to pupil results. 

As one source of evidence on the success of a seminar plan 
in the Teachers College of the University of Cincinnati, Pech- 
stein secured the responses of students to a questionnaire.” 
In seeking to determine the value of the services of the Edu- 
cational Research Bureau of Ohio State University, Charters °% 
used an interesting variety of measures. He noted the ob- 
jective accomplishments of the Bureau in terms of productions 
of the members, the number of Bulletins distributed, the num- 
ber of persons served through using the special library of the 
Bureau, the opinions of students and faculty, the number of 
specific services rendered, etc. 

Somewhat analogous to this problem of institutional rating 
is the problem of evaluating the services of teachers. Is this to 
be done in terms of measured results, or in terms of the 


292'The North Central Association has had committees at work revising its standards 
of accrediting for both secondary schools and higher institutions. Six different accrediting. 
organizations jointly undertook the study of appraising secondary schools, and this work 
is at this writing under way. See the following references: 

“Toward the Reformation of Standards for Secondary Schools,” School Review, XLI 
(December, 1933), 721-5. 

George E. Carrothers, “Associations United to Study High School Standards,” Nation's 
Schools, XIV (October, 1934), 31. 

Walter Crosby Eells, “Codperative Study of Secondary School Standards,” School and 
Society, XLII (October 19, 1935), 544-5. 

For other references on the revision of high-school standards, see the topic, ‘“Co- 
operative Study of Secondary School Standards Committee,” in the Education Index, 

293 The revision of standards for higher institutions by the North Central Association 
may be traced in the following references: 

G. F, Zook, “Work of the Committee on the Revision of Standards for Higher Insti- 
tutions in the Association,” North Central Association Quarterly, VIL (December, 1932), 
291-5. 

George A. Works, “New Basis for the Accteditment of Higher Institutions,” Educa- 
tional Record, XVI (July, 1935), 321-30. 

Evaluation of Higher Institutions: A series of monographs by the Committee on Re- 
vision of Standards for Higher Institutions of the North Central Association. Published 
by the University of Chicago Press. See for example, Vol. 7 by John D. Russell and 
Floyd W. Reeves, on Finance, 1935. Pp. 133. 

See Russell, footnote 180. 

For current references, consult the Education Index, topic: North Central Association 
of Colleges and Secondary Schools, sub-topic: Committee on Revision of Standards for 
Higher Institutions. 

294 L. A. Pechstein, “University of Cincinnati Esperimental Program in Seminar In- 
struction,” Tke Application of Research Findings to Current Educational Practices, pp. 
34-45. American Educational Research Association, Official Report of the 1935 meeting. 
Washington: National Education Association, 1935. Pp. 273. 

295 W, W. Charters, “Measures of Research,” Educational Research Bulletin, XI 


(October 12, 1932), 309-20. 
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teacher’s qualifications and characteristics? The use of tests 
for evaluating instruction was discussed early in the preceding 
chapter. The rating of teacher activities was discussed in con- 
nection with “Rating Scales.” Betts 2° gives an analytical dis- 
cussion of the difficulties and the possibilities of evaluating 
teaching by studying teachers’ traits, on the one hand, and by 
studying teachers’ procedures on the other. As in other cases, 
we should probably look to each line of evidence for the light 
which it may contribute toward a complete picture. 
Concluding statement. This chapter has continued the 
treatment of normative-survey methods and deals with the 
interview, observation, and survey-appraisal types. Survey test- 
ing, questionnaire inquiries, and documentary frequency studies 
were discussed in the preceding chapter. All of these methods 
are directed towards ascertaining what the prevailing condi- 
tions are, and how prevalent they are. That is, one may be 
interested in the average ability of pupils, the general practices 
of superintendents, the relative frequency of various practices 
with respect to report cards, the conditions found in the homes 
of pupils, the extent to which pupils engage in certain activities 
in different classrooms, the average estimate of the merit of 
a curriculum, the relative standing of state school systems, etc. 
The treatment of the normative-survey method in these two 
chapters has not included all of the possible procedures for 
gathering data, but has dealt with those that are regarded as 
most important. Other modes of gathering data, which have 
not been explicitly treated are: (1) collecting specimens for 
study; *°? (2) observing field objects and conditions in situ, 
such as inspecting school buildings to determine the number 
that possess certain characteristics, or taking photographs of 
them; (3) making physical measurements of inanimate objects, 


296 Gilbert L. Betts, “Problems in the Measurement of Teaching Ability,” The Appli- 
cation of Research Findings to Current Educational Practices, pp. 166-73. Official Report 
of the 1935 Meeting of the American Educational Research Association, Washington: 
National Education Association, 1935. Pp. 273. 

297 This procedure was referred to only incidentally in connection with documentary 
frequency studies, such as Heck’s; for another example, see George E. Hill, “The Report 
Card in Present Practice,” Educational Method, XV (December, 1935), 115-31. 443 
report cards from school systems were collected and examined. 
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such as school sites, school seats,” room temperature; and 
ventilation; 2% (4) making physical measurements of chil- 
dren; ®°° (5) copying data from original records; *% and (6) 
securing informal, personal records and reports made upon 
request, such as diaries, autobiographies, personal experiences, 
introspective accounts, and other recollections.* Illustrations 
of these techniques may be found in the references just cited. 
All of these procedures and possibly others may be used in 
normative-survey studies. They may also be used in other types 
of research, and certain of them are referred to in other 
chapters. 

The range of studies coming within the scope of normative- 
survey research is large. In field work, the bulk of the studies. 
fall within this class. The type of information yielded by nor- 
mative-survey investigations is in wide demand by practical 
workers. This type of approach does not produce fundamental 
Jaws, other than laws of distribution, and it is not concerned to 
any appreciable extent with analysis of cause and effect. It is, 
however, a basic type of approach in quantitative research, and 
all of the various techniques which it utilizes are likely to be 
employed as component steps in the more complex types of re- 
search, such as the experimental method, and the genetic, case 
study, correlational, and other methods which emphasize re- 
lationships. A thorough acquaintance with the procedures ap- 
propriate for normative-survey research will therefore afford 
an important background for the undertaking of research by 


other methods. 
Tn this text research is classified on the basis of the purpose- 


208 Henry Eastman Bennett, School Posture and Seating. Boston: Ginn and Co., 1928. 
Pp. 323. 

oo Sco the studies referred to by H. W. Schmidt, “Heating, Ventilation, and Sanita- 
tion in School Buildings,” Chapter II, pp. 344-61, in “The School Plant,” Review of 
Educctional Research, V (October, 1935), No. 4- 

200 See the studies referred to by Helen L. Dawson and George D. Stoddard, ‘Physical 
Growth from Birth to Puberty,” Chapter III, pp. 130-49, in “Mental and Physical De- 
velopment,” Review of Educational Research, IIT (April, 1933), No. 2. 

301 This procedure has been assumed in the discussion of index numbers; see also 
the first technique (listed by Caswell) in the discussion of school surveys. 

02 Data of this type were referred to in connection with a study by Blumer (footnote 


174 of Chapter VII). 
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ful structures of the entire undertaking, rather than on the 
basis of techniques of gathering data. In the historical method, 
which was described first, one seeks to describe conditions as 
they existed in the past, sometimes attempting to account for 
them and to draw practical generalizations from them which 
may be applied to the present or future. In the normative- 
survey method, the purpose is to describe conditions (including 
human-response tendencies, such as ability, beliefs, values, etc.) 
as they exist. Although one will draw inferences of practical 
significance from such studies, he does not seek primarily to 
account for the conditions observed. 

In the experimental method, which is discussed in the fol- 
lowing chapter, one seeks to discover the relation between 
changes in one factor and consequent changes in another vari- 
able (the result), in order to deduce generalizations concern- 
ing causes and effects. The experimental method is one of the 
classical methods of science. In Chapter X, methods of studying 
more complex cause-and-effect relationships are discussed. 
These methods are especially appropriate for the study of social 
phenomena, where both the causation and the consequences are 
highly complex and cannot be observed as “single” variables. 


PROBLEMS AND EXERCISES 


1. Some member of the class who is interested in attitude testing 
should try out the making of an attitude scale in a small way within 
the class. Select some topic of general current interest to the class, 
prepare a number of statements (say 20 or 30), and have about ten 
members of the class rank these. An analysis of the results should be 
discussed before the class. 

2. What types of data-gathering procedures might be appropriate 
at one time or another in the process of “job analysis,” as an approach 
to curriculum-making? Consider several different fields of training, and 
prepare a report illustrating the types of data and the procedures that 
would be appropriate for making the analysis. (See W. W. Charters, 
Curriculum Construction. New York: The Macmillan Co., 1923. See 
also, Hollis L. Caswell and Doak S. Campbell. Curriculum Develop- 
ment, Chapter X, “Selection of Subject-Matter,” pp. 248-90. New 
York: American Book Co., 1935.) The assignment may be varied as to 
field or approach. 
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3. A committee of persons interested should judge the relative in- 
telligence of the twelve children pictured by Goodenough and Ander- 
son, Experimental Child Study, insert, p. 224. Follow through their 
experiment No. 24, pages 231-3. Report to the class the correlations 
found (see their pages 133-6). 

4. Spend about two hours selecting and defining traits for a per- 
sonality-rating scale. Report to the class why you rejected some 
characteristics and retained others. The scales of various persons or 
committees should be discussed in class. 

s. Interested members of the class should secure data on the socio- 
economic status of some group of pupils in a convenient school, in 
accordance with one of the experiments outlined in Goodenough and 
Anderson, Experimental Child Study, pp. 234-44. 

6: Let several members of the class independently score a con- 
venient school building (perhaps the one in which the class meets), 
and compare the results of different scorers. 

7. It is difficult to secure practice in interviewing without drawing 
heavily upon the time of others—and perhaps intruding upon their 
personal affairs. One may get a certain amount of practice by inter- 
viewing other members of the class, perhaps about the community in 
which they grew up, political views they are willing to express, diffi- 
culties they have had in school, etc. Appropriate topics will vary with 
the individuals. Also, one may interview elementary- or high-school 
children concerning their difficulties and modes of work; such contacts 
must be made with caution and tact and with the approval and ad- 
vice of a regular teacher or counsellor of the child. 

8. Analyze the data in any school-survey report approved by the 
instructor, and list the techniques by which such data were probably 
secured. 

ọ. Examine the data in a state school-survey report, and draw up 
five to ten schedules (questionnaires or report blanks) that might be 
used to gather some of the data presented. 
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CHAPTER IX 


THE EXPERIMENTAL METHOD 


Introduction. Earlier chapters have dealt with two major 
approaches in attacking educational problems: analysis and 
interpretation of the evidence and experience of the past, in 
historical research (Chapter VI), and the canvass of practice 
and current conditions or the establishment of norms or central 
tendencies, in normative-survey research (Chapters VII and 
VIII). The present discussion is concerned with a third major 
method, experimental research, probably the most important 
of all from the strictly scientific point of view. Experimental 
research, whether conducted in classroom or laboratory, in- 
volves an attempt to control all essential factors save a single 
variable which is manipulated with a view to determining and 
measuring the effect of its operation. Obviously this procedure 
distinctly is different from the historical and survey methods. 

In the subsequent discussion, first the historical development 
of the experimental method is traced briefly. Next the experi- 
mental method is defined in terms of the law of the single vari- 
able and the control of observation and other pertinent factors. 
Principles of group experimentation (one-group, parallel-group, 
and rotation-group) are outlined, with considerable emphasis 
on procedures used to equate groups. Consideration is given to 
laboratory experimentation where conditions may be more care- 
fully controlled, with individual subjects and needed apparatus 
available, than in the classroom. The chapter ends with an at- 
tempt to evaluate the present contribution and to determine the 
future possibilities of the experimental method. 

Brief history of the experimental method in psychology 
and in education. In the past lie a number of great experi- 
ments, the conditions and consequences of which history and 
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the historical method seek to reveal for the guidance of present- 
day educational and psychological workers. The preparation 
for experimental psychology was philosophical psychology in 
the work of Aristotle, Descartes, Leibnitz, Locke, Berkeley, 
Hume, Hartley, James Mill, John Stuart Mill, Bain, Herbart, 
and Lotze. The actual founding of an experimental psychology 
is represented by the work of Weber, Fechner, Helmholtz, 
Wundt, Brentano, Stumpf, and G. E. Miiller, with later con- 
tributions by Ebbinghaus, Kiilpe, Mach, Avenarius, Titchener, 
Lipps, Galton, James, Hall, Ladd, Scripture, J. M. Baldwin, 
Jastrow, Sanford, Miinsterberg, Royce, Cattell, Thorndike, Ter- 
man, Judd, Woodworth, Dewey, and other workers currently 
interested in experimental procedure as applied to the problems 
of psychology and education. Space is not available here for a 
statement of the contributions made by these workers to the 
experimental method. Such information is readily accessible in 
a number of excellent sources.* 

In 1883 G. Stanley Hall “founded” at Johns Hopkins Uni- 
versity “the first psychological laboratory in America,” al- 
though James for several years previously had used space for 
experimental instruction at Harvard.” In 1887 Hall began pub- 
lication of the American Journal of Psychology and in 1891 the 
Pedagogical Seminary, the first psychological journals in Amer- 
ica, in which the early experimental studies appeared. He 
played a leading réle in planning the American Psychological 
Association and served as its first president in 1892. 

Controlled experimentation in education is indeed recent in 
origin, but in the pages of history one may find examples of 
attempts centuries ago to solve thought questions in education 
by means of trial and observation of results: Vittorino da Feltre 

1E. G. Boring, History of Experimental Psychology. New York: The Century Co., 
1929, Pp. xvi + 700. Emphasizes European psychology and the work of individual psy- 


chologists; comprehensive, but not elaborate as to recent work, 

J. C. Flugel, A Hundred Years of Psychology, 1833-1933. New York: ‘The Macmillan 
Co., 1934. Pp. 384. 

Gardner Murphy, An Historical Introduction to Modern Psychology, New York: Har- 
court, Brace and Co., 1929.. Pp. xviii + 470. Deals chiefly with recent work and is or- 
ganized largely by topics rather than chronologically. 

2E. G, Boring, op. cit., pp. 506-7. 
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(1378-1446) and Wolfgang von Ratke (1571-1635) in devising 
methods of teaching; Basedow’s (1723-1790) application of the 
theories of Comenius and Rousseau; Pestalozzi’s (1746-1827) 
schools at Stanz, Burgdorf, and Yverdun; and Herbart’s (1776- 
1841) practice school in connection with his pedagogical 
seminar.” Certain pioneer experimental schools in this country 
attempted to evaluate pedagogical theory by trial under actual 
working conditions: the Oswego Primary Teachers Training 
School established by Edward A. Sheldon in 1861, Francis W. 
Parker’s experimental school inaugurated in Chicago in 1883, 
and the Laboratory School of the University of Chicago 
founded by John Dewey in 1896. 

Early attempts to use the experimental method in education 
were impeded by inadequate control of the educative factors 
involved and by the lack of instruments for measuring pupil 
capacity and achievement. The investigations of Rice between 
1894 and 1897 marked the development of the concept of con- 
trol of experimental conditions. The work of Galton, Cattell, 
Binet, Terman, and Otis provided measures of pupil ability for 
use in securing groups equivalent in terms of intelligence. Stone, 
Thorndike, Courtis, Hillegas, Buckingham, and Ayres pio- 
neered the movement toward development of instruments for 
measuring pupil achievement, which have been used extensively 
in educational experimentation. It is only natural that the de- 
velopment of educational tests stimulated interest in experi- 
mentation under school conditions. Statistics has made its con- 
tribution to the experimental method through the work of 
Galton, Pearson, Yule, Spearman, Kelley, McCall, and others 
in terms of: (1) correlation techniques used to determine the 
validity and reliability of intelligence tests (employed to se- 
cure equivalence of groups) and of educational tests (used to 
measure gains in achievement); (2) statistics of errors (es- 
pecially the “error of a difference” formula, useful in the in- 

8W. S. Monroe and M. D. Engelhart, Experimental Research in Education, Chapter I. 
University of Illinois Bulletin, Vol. XXVII, No. 32, Bureau of Educational Research 


Bulletin, No, 48. Urbana, Ill.: University of Illinois, 1930. In the discussion of the ex- 
perimental method the present writers are much indebted to this excellent source. 
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terpretation of differences in gains in experimentation); (3) 
the “experimental coefficient” as a criterion for testing the 
statistical significance of a difference, etc.* 

Definition and characterization of the experimental 
method. Educational experimentation should be defined more 


specifically : * 


A child’s achievement from a period of learning is the resultant 
of several educative factors. “Experimentation” is the name given to 
the type of educational research in which the investigator controls the 
educative factors to which a child or group of children is subjected 
during the period of inquiry and observes the resulting achievement. 
... In the simplest type of educational experiment the investigator 
seeks to evaluate the influence of some one educative or “experi- 
mental” factor on a single group of children. He must start the ex- 
periment with some measurement of the initial attainment of the chil- 
dren in the trait or ability to be influenced. He then subjects the group 
to the experimental factor, such as a particular type of drill material 
in arithmetic, for the duration of the experiment. At the end, the in- 
vestigator applies a final test for the purpose of determining the gain 
in achievement that has resulted from the application of the experi- 
mental factor. 


Expressed in the language of logic, the experimental method 
represents Mill’s canon or principle of research which is known 
as the method of difference ; that is, to note the effect of a single 
variable applied to one situation or group, but not applied to 
a comparable situation or equivalent group.° 


Tf an instance in which the phenomenon under investigation occurs, 
and an instance in which it does not occur, have every circumstance in 
common save one, that one occurring only in the former, the circum- 
stance in which alone the two instances differ is the effect, or the cause, 
or an indispensable part of the cause, of the phenomenon. 


However, the interests and practical needs of the reader, as 
well as space limitations, render it inappropriate to enter into 
any extended discussion of the logic and philosophy of the 


4 Ibid, 


5 Ibid., pp. 15-6. 
6 John Stuart Mill, A System of Logic, Book III, Chapter VIII. New York: Long- 


mans, Green and Co., 1872 
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experimental method, since good references are available.’ It 
must be admitted, too, that the language of the logician and 
philosopher at times seems unnecessarily involved and abstract 
to be of most service in actual field situations. As a matter of 
fact, Mill’s canon represents an ideal assumption which prob- 
ably is never actually fulfilled in a concrete situation, certainly 
not in dealing with human beings in the social and biological 
fields where so many uncontrollable variable factors are in- 
volved, In such areas it is difficult indeed to conceive of differ- 
ent situations with all conditions absolutely identical except 
one circumstance or factor the effect of which is to be measured. 
Even in the field of physics it is being recognized that a great 
deal of previously unsuspected variability now exists. There- 
fore, early in this discussion it must be frankly recognized that 
the so-called law of the single variable is a theory and not an 
accomplished fact in any given educational investigation of the 
experimental type. Later in this chapter, numerous illustrations 
will be given of difficulties in controlling variables and of falla- 
cious conclusions based on the assumption of only a single 
operating variable. 

It follows from the definition of educational experimentation 
given earlier in this chapter that the major procedure involved, 
just as in experimental psychology and in all experimental 
science, is that of controlled observation, with an attempt to 
hold all factors constant except the single variabie the effect of 
which is to be measured. Naturally, experimental procedure is 
considered the classical method of science and frequently has 
been urged as the ultimate methodological goal of any system 


7J. L. Childs, Education and the Philosophy of Experimentalism. New York: The 
Century Co., 1931. Pp. xx + 264. Deals with the philosophical and educational ideas of 
Pierce, James, and Dewey. 

E. L. Clarke, The Art of Straight Thinking, Chapter VII. New York: D. Appleton 
and Co., 1929. 

M. R. Cohen and Ernest Nagel, An Introduction to Logic and Scientific Method, Chap- 
ter XIII. New York: Harcourt, Brace and Co., 1954. 

Columbia Associates in Philosophy, Introduction to Reflective Thinking, Chapter IV. 
Boston: Houghton Mifflin Co., 1923. 

D. S. Robinson, Illustrations of the Methods of Reasoning. New York: D. Appleton 
and Co., 1927. Pp. xiii + 346. 

F. W. Westaway, Scientific Method, Chapter XVII. London: Blackie and Son, Ltd., 
1924. 
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of investigation, since, with conditions artificially controlled, 
stimuli may be presented and reaction secured without awaiting 
the occurrence of events in their natural environment over a 
relatively long period of time (as is necessary in history).® 

However, it has been pointed out above that experimental 
procedure has distinct limitations in the social and biological 
fields where it is so difficult to control variable factors. Further- 
more, one may be more interested in a careful analysis of the 
interrelationships of various factors as they operate normally 
under conditions that are reasonably typical than in using a 
procedure which may do violence to a natural situation by at- 
tempting to hold all factors save one constant. A method, 
described in Chapter X, with its analytical techniques of par- 
tial and multiple correlation, makes it possible to study the 
effect of a number of factors varying jointly. The experimental 
and correlational methods are complementary, although the 
former is more limited with respect to the type of problem 
which may be attacked. However, the significance of any in- 
vestigational method, whether historical, normative-survey, 
experimental, correlational, or otherwise must be determined 
relatively in the light of the purpose and problem involved 
rather than in absolute terms. 

Confusion frequently arises concerning the relation of un- 
controlled observation, and testing or measurement, to ex- 
perimental procedure. Uncontrolled observation and record- 
ing of the spontaneous behavior of children in a variety 
of situations, with no attempt to condition the type of re- 
sponse, are normative-survey or genetic rather than experi- 
mental in character; for example, a frequency count of the 
words used orally by one thousand six-year-old children in 
their homes and schools. However, uncontrolled observation 
shades into experimental procedure when external conditions 
surrounding the child are controlled by eliminating the presence 
of the observer (through use of a “one-way vision” screen), 


8J. E. Anderson, “The Methods of Child Psychology,” A Handbook of Child Psy- 
chology, (Second edition, revised.) Chapter I. Edited by Carl Murchison. Worcester, 
Mass.: Clark University Press, 1933. 


= 


+ 


488 EDUCATIONAL RESEARCH 


selection and conditioning of the child’s companions, use of an 
“infant cabinet,” or even by maintaining environmental con- 

h ditions such as teacher, children, and classroom constant.? 
Thus, most essential factors (teacher, pupils, etc.) might be 
kept constant, except the passage of time, and the accompany- 
ing growth or maturation observed at intervals and recorded in 
the form of a growth or learning curve. Since many such 
studies are of a genetic character, involving an analysis of the 
causal factors operating to produce change by way of growth or 
maturation, they are reserved for discussion in Chapter X. 
Some workers prefer to think of such genetic investigations, 
when extended over a long time span, as closely related to or 
involving the historical approach. 

The overlapping of research techniques may be shown by 
| another example, drawn from the testing field, and the relation 
_ of testing to experimentation indicated. If intelligence testing 
or achievement testing is done for the purpose of ascertaining 
| the capacity or accomplishment of the child in relation to other 

children or to a norm, the procedure is normative-survey rather 
than experimental. The testing done to equate groups of chil- 
dren for experimental purposes and to measure results at the 
end of the investigation may be considered of a survey char- 
acter, but fits into the larger purpose or pattern of the experi- 
ment and properly may be regarded as a definite part of the 
experimental investigation. To use a different illustration, if 
an intelligence test is administered year by year to a given 
group of chiidren who have lived under normal and uniform 
(controlled) conditions during the time involved, with matu- 
ration as the single variable operating, the resulting curve of 
mental growth is the product of an experimental procedure. 
A study of the accompanying age norms and of the factors 
operating to produce such growth takes on genetic and causal 
characteristics. Parenthetically, the great difficulties present in 

9G. D. Stoddard and Beth L. Wellman, Child Psychology, Chapter II. New York: 
The Macmillan Co., 1934. 


10 For example, Catharine M. Cox, Genetic Studies of Genius, Vol. II. Stanford Uni- 
versity, Calif.: Stanford University Press, 1926. Pp, xxiv + 842. 
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seeking to control or to make uniform the conditions surround- 
ing the child over any extended period of time should be recog- 
nized at this point and reserved for later consideration in this 
chapter. Of course, viewed in a broad way, it is possible to 
think of the administering of any standard test (psychological 
or achievement) as an experimental procedure, since certain 
factors such as directions and classroom conditions presumably 
are controlled for purposes of observation of the performance 
of the child. The experimental factor may be thought of as the 
instruction which the pupil undergoes between testing periods, 
the effect of which is measured. However, this is a type of study 
which more appropriately than here has been discussed in 
Chapter VII in connection with normative-survey procedures. 

The point of the preceding discussion is that there is no 
sharp line of demarcation between so-called observational, 
survey-testing, genetic, and experimental studies. Students who 
view the definition of the experimental method more broadly 
than is represented in the subsequent portion of the present 
chapter do no actual violence to the principles of controlled 
observation and the manipulation of an experimental factor by 
considering many observational, repeated survey-testing, and 
growth investigations as types of experimentation. However, 
the point of view of the authors and the functional and logical 
organization of research methods are represented more ade- 
quately by assigning these studies to the appropriate chapters 
mentioned above. 

Obviously, the accuracy of observation has been greatly in- 
creased by standard tests, mechanical controls such as those 
used in the psychology laboratory (chronoscope, tachistoscope, 
color mixer, galvanometer, spectroscope, photographic equip- 
ment, etc.), repeated observations, and statistical techniques.* 
However, much important pioneer work has been done without 
elaborate equipment for experimentation, which indicates that 
the intellectual curiosity and ingenuity of the investigator are 


11A. G. Bills, General Experimental Psychology. New York: Longmans, Green and 
Co., 1934. Pp. xii +°620. The introduction includes a discussion of the assumptions, 
methods, and field of experimental psychology. 


Èi 


490 ' EDUCATIONAL RESEARCH 


more important than ornate laboratories." Galileo climbed the 
leaning tower of Pisa to make a practical test of the validity 
of the then common belief that a heavy body falls faster than 
a light one. While in his bath, Archimedes is said to have dis- 
covered that when a body is immersed in a fluid, it is buoyed 
up by a force just equal to the weight of the fluid displaced. 
Helmholtz said that after he had been working on a problem 
for some time, happy ideas for its solution usually came un- 
expectedly like an inspiration, and never while he was at his 
working table. While on foot in the mountains of Switzerland, 
far from his laboratory at Columbia University, Michael Pupin 
invented the Pupin coil, of such importance to the long-distance 
telephone. Darwin was able to recall the exact spot in the road, 
while riding in his carriage, when his theory of evolution oc- 
curred to him. James Watt was walking on a fine Sunday after- 
noon when he invented the condensing steam engine; later he 
had to make the steam cylinder and piston out of a surgeon’s 
large brass syringe. When Pasteur went to his home town to 
study the diseases of wine in an effort to save this industry, he 
had to use the crudest of equipment: an old room that had been 
a café, apparatus made by the carpenter and tinsmith of the 
village, an open charcoal brazier with a pair of bellows instead 
of a gas burner, and water carried from the town pump. Yet 
here pasteurization was discovered, and .the wine industry of 
eastern France saved. Charles Goodyear, experimenting on rub- 
ber in a New England kitchen, happened to bring a mixture 
of sulphur and rubber into contact with the hot stove, and thus 
discovered how to vulcanize rubber. He made some of his 
experiments in this field while confined in a Philadelphia jail 
as a common debtor. 

Certain of the foregoing illustrations suggest that much 
of the success of experimentation depends on close and careful 
observation rather than on elaborate equipment, as is the case 
in any area of investigation.** Darwin attributed many of his 

12T. A. Boyd. Research, pp. 8, 27, 44, 55-62. New York: D. Appleton-Century Co., 


1935. 
13 Ibid., pp. 82, 87-93. 
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own accomplishments to an ability to notice things which easily 
escape attention, and to observe them carefully. Franklin no- 
ticed that, contrary to accepted opinions, storms traveled in the 
opposite direction to the wind, or that northeasters sometimes 
reached Philadelphia before Boston; in this way modern 
meteorology is said to have originated. Although accident fre- 
quently plays a part in discovery, little benefit to the cause of 
science results unless intelligent observation is also involved. 
The French physicist, Becquerel, found that a little radium 
carried in his pocket made a spot on his skin like an ulcerated 
sore, thus suggesting the use of radium in the treatment of 
cancer. Edison was rolling between thumb and forefinger a 
piece of lampblack mixed with tar (prepared for use in his tele- 
phone transmitter), when it occurred to him that a thin spiral 
of this material might be a good filament for the incandescent 
lamp on which he had worked for two years. When tried, it 
worked, although the material was not good enough for fila- 
ments, but Edison was started toward a successful carbon- 
filament lamp. Prospectors who had been hunting for radium 
in Canada pitched camp in the evening after a fruitless day. 
Their fire melted away the snow, revealing moss that appeared 
to be eaten away by some agency. Curiosity and intelligently 
directed observation showed that a deposit of radium was 
beneath their feet. 

General principles of group experimentation. Of course 
the conditions of the psychological and experimental-educa- 
tion laboratories make possible the use of elaborate techniques 
and complicated apparatus appropriate to experimentation with 
individual subjects. Most educational experiments in the field 
are conducted in the classroom where it is necessary to work 
with groups of children rather than with individuals. McCall’s 
description of group methods of experimentation in education 
has been widely quoted: the one-group method, the equivalent 
or parallel-group method, and the rotation-group method. 


14W. A. McCall, How to Experiment in Education, Chapter Il, New York: The 
Macmillan Co., 1923. 
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The one-group technique. A one-group experiment has been 
conducted when one thing, individual, or group has had applied 
to or subtracted from it some experimental factor or factors and 
the resulting change or changes determined or measured. For 
example, if a group of pupils takes equivalent forms of a stand- 
ard reading test in order to determine the effect on their scores 
of varying mental attitudes induced by a different mental-set 
for each form of the test, a one-group procedure is involved. 
The pupils may take one form, using the regular printed direc- 
tions accompanying the test. Scores on this form serve as a 
norm against which to check performance on other equivalent 
forms of the test when the experimenter has sought to induce 
specific mental attitudes such as encouragement or discourage- 
ment. In general, it is desirable, and even necessary in terms 
of significance of findings, to know the normal or expected 
performance of the group as a basis for comparison with results 
produced by application of the experimental factor. Thus, if 
a first grade is subjected to an experimental method of teach- 
ing reading for the entire school year, the results can be inter- 
preted only in terms of normal or expected performance for a 
group of this given intelligence and home background while 
following a customary teaching procedure. However, it must be 
recognized that any attempt to estimate the performance of 
a given group as the result of a procedure never actually tried 
with the group in question is fraught with difficulties and 
possibilities for error. 

The foregoing description of one-group experimentation sug- 
gests that it is the simplest of all experimental procedures and 
the most feasible for classroom use, although probably the 
least valid. With the factors of pupils, teacher, and school 
setting constant, the only variable present is the experimental 
procedure, and of course such change as takes place in the 
group or teacher with the passage of time and with maturation. 
These changes in the pupil which take place when different 
phases of the experimental factor are applied in sequence over 
a period of time involve at least four defects in the one-group 
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method.* As different forms of a standard test in reading are 
administered in the first grade at intervals of, let us say, one 
month or even one week, with an attempt to induce varying 
mental attitudes: (1) gains may be greater between earlier 
forms, since the typical curve of learning shows more rapid 
acceleration in the earlier stages of practice; (2) practice effect 
may operate in such a way as to increase scores on later forms 
of the test; (3) an attitude such as confidence or discourage- 
ment or a method of attack or study may carry over from one 
form to another; (4) the fact that the child is a given number 
of days, weeks, or months older, with a corresponding level 
of maturity and achievement reached, may affect performance 
or the significance of a gain on later tests.‘° For this hypo- 
thetical example, reading in the first grade has been chosen, 
so as to make the possible limitations of the one-group method 
especially conspicuous. 

In summary, it may be said that the essentials of the one- 
group method are: (1) an initial test, the application of an 
experimental factor, and a check on results; (2) a second pre- 
liminary test, the application of another experimental ‘factor, 
and a check on results; and (3) as many other cycles of pre- 
liminary testing, experimental procedure, and end testing as 
are desired. 

The parallel-group technique. The parallel-group procedure 
represents an attempt to overcome the limitations of the one- 
group method, since two or more groups, as nearly equivalent 


15H. H, Abelson, The Art of Educational Research, pp. 141-2. Yonkers-on-Hudson, 
N. Y.: World Book Co., 1933. 

16 Edward A. Lincoln, “The Equality of Units in Educational Measurement Com- 
pared,” Journal of Educational Research, XXII (October, 1930), 197-202. The point of 
view is aptly stated as follows: 

“An inch added to a man’s stature is quite a different thing in value and meaning 
from an inch added to the end of his nose. Furthermore, the inch in stature has not al- 
ways the same meaning. As already pointed out, it may represent only a month’s growth 
in the days of infancy, while in late adolescence two years may be necessary to acquire 
it. Further still, an inch in stature means less to the individual of average height than it 
does to the short man who without it will be noticeable as a runt, or to the tall man 
who will become over-conspicuous because he has it. This difficulty is present in measur- 
ing inanimate things as well as living organisms. An inch on a transatlantic cable is very 
different frem an inch on the weight beam of a sensitive balance. A pound of granite is 
different from a pound of gold. Even time units do not avoid this difficulty; a year in 


college is different from a year in jail.” 
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as possible in all respects, are used at the same time. Under 
carefully controlled conditions, only a single variable is manip- 
ulated, namely, the factor which the experimenter varies for 
the two groups, whose effect he attempts to determine. To one 
group may be applied the experimental factor, with the parallel 
group serving as a control for comparative purposes, following 
a customary or non-experimental procedure. Or, different 
phases of the experimental factor may be applied to the two 
equivalent groups, although in such an instance, a third parallel 
group, following a non-experimental plan, is desirable to serve 
as a control for purposes of comparison. For example, one of 
two equivalent groups may take a standard test under condi. 
tions of discouragement, induced by the experimenter, while 
the second group uses only the customary directions accom- 
panying the test. Or, the second group may take the test under 
a condition of encouragement and praise, which is another 
phase of the experimental factor of mental attitude; however, 
in this event a third parallel group using only the printed 
directions accompanying the test is desirable as a control. Of 
course’the greatest difficulty in this method is to equate groups 
adequately. 

It has been emphasized especially that indiscriminate use of 
the control-group technique impedes real progress in education 
due to the fact that: (1) invalid results become perpetuated 
in the literature where they influence the uncritical reader, and 
(2) the methodology of research may suffer through over- 
standardization of the control-group method. Suggestions for 
safeguarding this type of experimentation are as follows: * 


1. Exercise care in organizing the investigation so as to give the 
control group a fair chance so far as the experimental factor is con- 
cerned, 

2. Be certain that the type of training given the experimental and 
control groups constitutes an important factor experimentally. 

3. Note various types of change and improvement which take place 
in the control group during the experimental period. 


17 W. A. Brownell, “Some Neglected Safeguards in Control-Group Experimentation.” 
Journal of Educational Research, XXVII (October, 1933), 98-107. 
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4. Examine critically the function and place of standard tests in 
the measurement of results. 

5. Make the measurement complete and comprehensive. 

6. Use sagacity and insight in the interpretation of measures. 


The rotation-group technique. The rotation method involves 
the reversal of the groups at intervals, in terms of the pro- 
cedures followed. This method is frequently used when parallel 
groups are not available or there is doubt concerning the 
equivalence of the groups, due to such factors as initiative, 
industry, or study habits which are very difficult if not im- 
possible to control. For example, the parallel groups might 
take a second form of the same standard test with an exchange 
of the mental attitudes induced by the experimenter. The 
groups may be reversed as often as desired by the investigator. 
This procedure is illustrated in the diagram which follows: 


Group A B 
Method} «7a siiwies girai x y 
Method. daai bosas y x 
Method sssi sessed enea, x y 
Methodiin ias + sis ni y x 


Again, three groups, not known definitely to be equal, might 
use at different times the following procedures, with the results 
for each plan combined and compared with the other methods 
at the end of the experiment. That is, the results for method « 
would be totaled for all three groups and compared with totals 
for methods y and z. However, even here, there must be reason 
to believe that the groups are equally capable of profiting from 
exposure to the same experimental factor. To use an extreme 
example, one would not compare groups of blind, deaf, and 
crippled children respectively. Obviously, there are other se- 
quences in which the different methods may be taken up by 
the three groups as they proceed from one phase of the ex- 
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perimentation to another, for example, Group A may begin 
with either y or z instead of x. 


A B Cc 
x y z 
y x 

x y 


The major possible limitation of the rotation method is the 
same as for the one-group method, carry-over effect from one 
experimental procedure to another, rendering it difficult to de- 
termine whether the score on a given form of the reading test 
mentioned above is caused solely by the attitude the experi- 
menter seeks to induce or in part by a combination of attitudes 
transferring from earlier forms of the test. It will be recog- 
nized that use of equivalent groups adds to the validity and 
usefulness of the rotation procedure. Other limitations (pre- 
viously listed) of the one-group method are represented in the 
rotation pattern (sketched above for the three groups) in that 
‘a given procedure was followed at three different time periods 
in the course of the experiment. The hypothetical examples of 
group experimentation given above are used only for illustra- 
tive purposes, with due recognition of difficulties present in 
attempting to induce and measure the effects of a given mental 
attitude, 

An illustration of a more intricate method of rotating pro- 
cedures, which aims to equalize practice effect and other similar 
variable factors, involves a ball-tossing experiment.’* 

Seventy-seven adult male students served as subjects. Sixty-six 
of these composed the six experimental groups, and eleven com- 
posed the control group. Each subject in the experimental groups 
was given sixty trials with each of the eleven instructions, or 
660 trials in all. About seventy-five minutes were required for a 

18 R. L, Hoke, “Factors Conditioning Efficiency in a Motor Skill,” Abstracts Graduate 
Theses in Education, 1927-1931, Teachers College, University of Cincinnati, pp. 135-44. 


Cincinnati, Ohio: University of Cincinnati, 1931. Also Journal of Experimental Psychology, 
XV (June, 1932), 316-30. 


THE EXPERIMENTAL METHOD 497 


subject to complete the experiment. In order to equalize prac- 
tice effects and other similar variable factors, the counter- 
balanced order of giving instructions was used. The experiment 
was organized in six cycles for six groups of subjects. In each 
cycle were eleven subjects to correspond with the eleven in- 
structions. Each group of subjects had its own sequence of 
instructions. All subjects went through the eleven conditions 
or instructions once, each in his own order. Subject 1 of the 
first group began with the first of the instructions (A), went 
through the second (B), third (C), and so on. Subject 2 began 
with the second (B) of the instructions, then took the third 
(C), and finally went through the first (A). This scheme was 
carried through for each group of subjects so that each instruc- 
tion appeared equally often in each serial position. The order 
in which instructions were given the first two groups is as 
follows: 
First Group of Subjects 


Subject Instructions 
I ABCDE EGH ITK 
2 BCDEFGHIJKA 
3 CDEFGHIJKAB 
4 D.E- EG, Bal JAB C 
5 E FsG Hal jek ALB C.D 
6 FGHIJKABCDE 
7 Gent jy KA SCDE 
8 HIJKABCDEFG 
9 IJKABCDEFGH 
10 JKABCDEFGHI 
Ir KABCDEFGHIJ 
Second Group of Subjects 
Subject Instructions 
I JIHBGKFACDE 
2 IHBGKFACDEJ 
3 fe G¢ KFA CD EJ 1 
4 BGEKFACDEJIA 
5 GKR EAC DEJT HB 
6 KFACDEJIHBG 
7 FACDEJIHBGE 
8 AED yA HB CKE 
à CPE PA B GRETA 
to DEJIHBGKFAC 
ir EJIHBGKFACD 


TABLE XIN 


ORDER OF MAZES AND CONDITIONS OF THE EXPERIMENT 


Ea — 
Order of Mazes 
First Second Third Fourth 
Subjects 
EEE TEE EIV ALE N I E L NE Aa N T rb? oll Sg: 
Na} EON My Se 
L|M/]HIN 
MCH | Nek 
H|N/|L{IM 
N L M|H 
EREM tee ON 
M|H N L 
H N L M 
N L M R 
L:M| H |N 
M|JHİNIİL 
H|N|L{M 
N | L M | H 
L M/|H|N 
M H N L 
HSIN S oe [eae 


N=no punishment; L= light; M=medium; H=heavy. 


THE EXPERIMENTAL METHOD 499 


Another excellent example of the counterbalanced procedure, 
used to equalize variable factors, is found in Table XIII 
which is to be read in the following manner: *° Subjects A and 
AA began the experiment by running maze I under a condi- 
tion (N) in which there was no punishment, then ran maze IT 
under light (L) punishment, maze III under medium (M) 
punishment, and maze IV under heavy (H) punishment. Sub- 
jects E and EE began the experiments by running maze IT 
under the normal (N) condition in which there was no punish- 
ment, then ran maze III under light punishment, maze IV 
under medium punishment, and maze I under heavy punish- 
ment. Two series of experiments, each involving sixteen sub- 
jects were conducted. 

A check-list for experimentation. A helpful check-list °° 
of items to be considered in experimental work, with special 
reference to group procedure, based on McCall’s discussion ** 
of the problem, is available. Although the various items vary in 
their significance for experimental work, according to the type 
of situation involved, they are given in some detail for the 
sake of a reasonable degree of completeness. 


EXPERIMENTAL METHODS 


1. In any experimental study the total net change in the trait or 
traits in question produced by irrelevant factors must be negligible, or 
the amount of such change must be measured and discounted by the 
application of a control experimental factor. 

2. The one-group experimental method is valid where the change 
produced by an experimental factor is not conditioned significantly by 


19 James Vaughn and C. M. Diserens, “The Relative Effects of Various sntensities of 
Punishment on Learning and Efficiency,” Journal of Comparative Psychology, X (Feb- 
ruary, 1930), 55-66. 

‘Also see E. S. Robinson, “Methods of Practice Equilibration,” American Journal of 
Psychology, XLI (January, 1929), 153-6. Discusses the following procedures: 

1. Method of completed practice 

2. Methods of uncompleted practice 

(a) Method of predicted learning 
(b) Method of the control group 
(c) Methods of counter-balanced order 

20H. H. Bixler, Check-Lists for Educational Research, pp. 15-21. New York: Teachers 
College, Columbia University, 1928. 

21 W. A. McCall, op. cit. 
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any preceding factor, and where the change effected by each experi- 
mental factor is measurable in equal units. 

3. The equivalent-groups method is valid where it is possible to 
equate groups. 

4. The rotation method is valid where the change produced by an 
experimental factor is not conditioned significantly by any preceding 
experimental factor. 

5. The experimenter should make plans for keeping an experi- 
mental log or record of events, with dates, and should include irrele- 
vant factors and unusual conditions. 


Tue SUBJECTS 


6. The experimenter should have a carefully thought-out plan for 
selecting the subjects, that is, the group to which the experimental fac- 
tors are to be applied. 

7. If it is proposed to use statistical formulas whose validity de- 
pends upon the assumptions of random sampling, the experimenter 
must take precautions to assure that these assumptions are met. 

8. When the composition of the population to be studied is known, 
the sample must be so selected as to include all the essential elements. 

9. When the composition of the population is unknown, the experi- 
menter should select at random sample groups and continue selecting 
groups until comparison of the samples taken reveals a definite tend- 
ency for variations in one direction to occur as frequently as variations 
in another direction. 

10. The method of sampling should be described in full so that the 
research may later be evaluated, 


The experimenter, in selecting subjects for the experiment, must 
check on the following points: 


11, The subjects must be appropriate to the experimental methods. 

12. The subjects must be appropriate to the tests to be used. 

13. The subjects must be appropriate to the experimental factors. 

14. The subjects should be representative as to age, sex, grade, in- 
telligence, etc. 

15. The experimenter should select subjects who may reasonably be 
expected to be available throughout the period of the experiment. 

16. The number of subjects must be sufficiently large to assure re- 
liability of the results. Sometimes a preliminary trial experiment is 
needed to determine the approximate number. 

17. The number of subjects must be sufficiently large to allow for 
‘losses or eliminations on account of absences from one or more tests, 
inability to pair, and other reasons. If there is a control group, it is 
usually desirable to select a much larger number of subjects for it than 
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will be finally needed, so that all members of the experimental group 
may be utilized in pairing. 


Tue MATERIALS OF EXPERIMENTATION 


18. The materials must be appropriate to the subjects. 

19. The materials must be appropriate to the experimental method. 

20. The cost of the materials must be within reasonable limits. 

21. The materials must be appropriate to the conditions under 
which the experiment will be conducted, such as the time available and 
the qualifications of the persons who are to use the materials. 

22. The materials must not present irrelevant or variable factors 
that might invalidate the experiment. 


Tue PLACE OF THE EXPERIMENT 


23. The place or places in which the experiment is to be conducted 
must be typical of the situations to which the results of the experi- 
ment are expected to apply. 

24. This representative character may be planned with reference to 
parts of a city, parts of a county or state, parts of the United States, 
etc. 

25. The place and its environment must be appropriate to the ex- 
perimental method and the experimental factors. 

26. The place or its environment must not introduce irrelevant or 
variable factors that might invalidate the experiment. 


Tue TIME or THE EXPERIMENT 
* 


27. The experimenter must determine whether the time of the day, 
of the month, oz of the year is a significant factor in the experiment. 

28. In deciding upon the time length of an experiment, the principle 
to be kept in mind is that one should aim to secure the maximum 
effect of the experimental factors with a minimum effect from irrele- 
vant or variable factors. 


Equatinc Groups 


When groups are to be equated, one or more of the following bases 
may be used: 


29. Random selection, that is, chance arrangement 

30. General ability 

31. Initial status in the experimental trait 

32. Composite of several test scores 

33. Rate of growth or change in the experimental trait 
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34. Multiple bases, such as chronological age, mental age, and ini- 
tial status 


CONTROL AND MEASUREMENT OF EXPERIMENTAL 
AND IRRELEVANT FACTORS 


„35. If tests are needed in the experiment, choose appropriate meas- 
uring instruments. 

36. The experimental conditions or situations must be appropriate 
both to the problem and to the subjects. 

37. The experimenter should have or obtain the authority necessary 
for the control of experimental conditions. 

38. The experimenter should measure the amount of the experi- 
mental factors as accurately as possible. 

39. The experimenter should measure the amount of change in the 
trait in question produced by the experimental factors. 

40. The amount of change in the experimental subjects due to ir- 


relevant factors should be eliminated, equated, or accurately measured 
and discounted. 


Among the constant irrelevant factors, which may or may not be 
significant, are the following: 


41. Maturing or growth of subjects. 

42. Bias of experimenter or assistants. 

43. Capability of assistants. 

44. Enthusiasm or unwillingness of assistants. 
45. Bias of the subjects. 

46. Differences in time allowance. 

47. Too short or too long a time allowance. * 


48. Differences in transfer of knowledge, enthusiasm, and the like, 
among subjects. 


49. Bias of the tests. 


50. Physical environment, such as temperature, home surroundings, 
and the like, 


51. Unnatural conditions, or lack of adjustment of the subjects to 
the situation. 


52. The experimenter must keep accurate, detailed, and dated rec- 


ords of the methods of applying the experimental factors, possible 
irrelevant factors, and other pertinent data. 


53. The research worker should determine whether any other 


method, such as partial correlation, may be used to supplement or 
check on the experimental method. 


Equating of groups and control of variable factors. It has 
been pointed out above that the most difficult problem in 
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controlled group experimentation is to insure the equivalence 
of parallel groups. Many gross errors have been made through 
ignorance and carelessness in attempting to form parallel 
groups, and in reaching conclusions based on comparisons be- 
tween unequal groups. To say that pupils should study Latin 
in high school in order to succeed in college, simply because 
there is a positive relationship between college marks and the 
amount of Latin studied in secondary school, is to neglect 
probable differences in intelligence between Latin and non- 
Latin pupils. To compare absences in s given secondary school 
for the periods, 1915-1918, 1925-1928, and 1930-1933, assuming 
that the pupils, teachers, and environmental conditions rep- 
resent three equivalent situations, violates virtually every prin- 
ciple of experimentation. Obviously, the pupils and teachers 
are not the same, even though the numbers are large enough 
to represent a random sampling of a relatively stable com- 
munity. The conditions in a war period (1915-1918) with easy 
employment for youth and an influenza epidemic, in the 
“boom” period of prosperity (1925-1928) with its effect on 
employment and school attendance, and in a depression period 
(1930-1933) with a different effect on employment and attend- ` 
ance, with possible strengthening of compulsory education 
laws in the post-war period, are factors rendering the three 
situations far from comparable. 

One would-be investigator who wished to compare two 
methods of teaching English in high school, in the first place, 
had only a dozen members in each group. The boys were placed 
in one group and the girls in another (a variable of sex). The 
girls were older tnan the boys by an average of nine months, 
and their mental age fourteen months higher. An attempt was 
made to compare the I.Q.’s of the two groups, rather than their 
mental ages, which of course was incorrect. In seeking to use 
other procedures, which to her were only names, pairing was 
attempted by matching the lowest pupil in one group with the 
lowest in the other group, etc., without regard to whether the 
scores were equal. The whole procedure suggests the tragedy, 
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ers in a given area should prove helpful in selecting the factors 
to be controlled. 

Although it is hardly possible to express in mathematical 
terms all of the extra-school factors involved, they should not 
escape the attention of the experimenter in planning the in- 
vestigation and in interpreting results. 

The items listed above emphasize actual measurement of 
the factors involved as a basis for equating groups. Some ex- 
perimenters assume that given groups are equivalent because 
they were formed as a result of the operation of chance, which 
is a very questionable procedure in view of the small numbers 
and selective factors involved in the school and community set- 
tings for most educational experiments in the field. 

Naturally, as many as possible of the foregoing factors 
should be used in forming parallel groups, rather than to de- 
pend solely upon a single measure such as intelligence. These 
factors may be combined into a composite score or considered 
separately in equating groups. If scores are combined, the 
weighting of the various traits must be considered carefully, 
since it is generally recognized that averages or composite 
scores may be the same for two quite different persons. If the 
traits are used separately in forming parallel groups, consider- 
able shifting and elimination of pupils usually take place be- 
fore satisfactorily equated groups result. 

This statement implies that it is a more refined technique 
to pair pupils in the parallel groups than to depend solely on 
a comparison of the central tendencies and variabilities of the 
groups in question considered as a whole. One of the best 
examples of pairing is the use of identical twins in certain of 
the psychological studies.** Averages may cover up differences 
which are clearly apparent when the more analytical method 
of pairing is used for equating groups and for consideration of 


24 Arnold Gesell and Helen Thompson, “Learning and Growth in Identical Infant 
Twins: An Experimental Study by the Method of Co-Twin Control,” Genetic Psychology 
Monographs, VI (1929), 1-123. 

G. D. Stoddard and Beth L. Wellman, Child Psychology, pp. 28-30. New York: The 
Macmillan Co., 1934, 


| 
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relative performance. The highest pupil in Group A, in terms 
of some appropriate combination of the various equating meas- 
ures employed, is paired with the highest pupil in Group B, 
if they are reasonably comparable, the second highest in 
Group A with his mate in Group B, etc. Of course, in each 
experiment a decision must be made with respect to the 
maximum difference allowable between the two members of a 
pair, in terms of the equating measures employed. Usually 
there are certain pupils in both groups who cannot be matched 
or paired closely, and for the purposes of the experiment their 
results may be disregarded, comparing only the performance 
of the paired individuals. Such a plan makes possible the use 
of classes without shifting pupils from their original sections. 

Not only should the central tendencies of the groups be 
considered, but the variability of traits as well. Differences in 
performance between groups with equal central tendencies 
may result when the members of one group are clustered 
closely around the mean or median, while the traits of the 
other group are distributed over a much wider range. Of course, 
the technique of pairing mentioned above insures a similar 
variability and central tendency of scores for the equivalent 
groups. A rather refined statistical procedure for balancing 
parallel groups in terms of both mean scores and variability 
among scores in the groups has been worked out so that only 
a minimum of cases needs to be dropped to equalize the 
groups.*° The statistical procedures to which reference is made 
here are discussed in Chapter XI. Psychological and achieve- 
ment tests, as data-collecting instruments useful in equating 
groups and in measuring results, are considered in Chapter VII. 

A good example * of the technique of equating groups, in- 
volving the pairing of pupils, is given in Table XIV. The 
paired group in the large class had the advantage in composi- 
tion (.17), average I.Q. (.53), and reading ability (.04). The 

25 P, J. Rulon and Charlotte W. Croon, “A Procedure for Balancing Parallel Groups,” 


Journal of Educational Psychology, XXIV (November, 1933), 585-90. 
26 Dora V. Smith, Class Size in High-Schoo! English, pp. 84-8. Minneapolis, Minn.: 


University of Minnesota Press, 194¢ 
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TABLE XIV 


AVERAGE I.Q., CHRONOLOGICAL AGE, AND ACHIEVEMENT SCORES OF THE PAIRED 
PUPILS IN THE LARGE AND THE SMALL CLASSES, SEPTEMBER 1926-1927 


Score 
Pair Size of | Average} C.A. 
Class 1.Q. Read- | Gram- | Compo-| Spell- 
ing mar sition ing 
De hes Small 136 163 128 77 6.1 130 
Large 142 157 128 85 5.6 127 
Bio aie Small 135 159 124 82 5.9 130 
Large 137 155 113 79 6.2 128 
Baas hele Small 131 165 108 90 5.7 133 
Large 127 161 110 84 Bg 134 
Y TE Small 125 168 109 61 5.5 III 
Large 122 167 108 63 5.1 119 
ENENGE Small 123 157 109 72 6.6 123 
Large 126 155 103 61 6.5 102 
(hyd Raney Small 123 167 114 69 4.2 102 
Large 123 162 IOL 67 52 121 
Batic Small 123 158 114 76 6.3 108 
Large Ira 159 IOI 46 5.3 126 
RA Small TI9 166 IIo 42 6.7 124 
Large 118 168 III 59 6.0 117 
E PATER Small 117 159 gr 74 5.1 128 
Large 120 159 80 76 5-5 124 
CONIA ESA Small 114 166 85 57 5.5 112 
Large 115 163 90 69 4.4 116 
LIS seats Small 114 169 100 82 ges. 125 
Large 114 173 85 70 5-9 125 


small class surpassed the mated group in age (.29 months), in 
grammar (1.52), and in spelling (.81). In no case did the dif- 
ference exceed the probable error of the means except in 
composition. From the standpoint of significance, the chances 
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TABLE XIV—Continued 
Score 
Pair Size of | Average} C.A. 
Class IQ. Read- | Gram- | Compo-| Spell- 
ing mar sition ing 
Matas’ Small 114 170 106 72 4.7 123 
Large I4 174 114 14 4.6 IIL 
ne Rarer ie Small 113 173 85 45 Sir 123 
Large 109 178 94 51 4.6 III 
T Onin Small 112 173 94 31 5.5 122 
Large 112 174 109 30 6.5 121 
Ra Small III 162 76 50 4.1 13I 
Large 113 171 75 55 4.7 116 
MOI ford NAL Small ng he 163 95 39 5.4 13I 
Large 109 164 80 45 5.1 I21 
KNs a ra Small IIo 168 71 77 4.0 113 
Large 114 163 9o 52 5.8 121 
BOs trae Small 105 175 77 30 3-4 69 
Large 108 178 95 66 4.5 I21 
Toas Small 103 167 88 71 4.3 127 
Large 108 158 83 70 4.7 116 
a AE Small 99 186 81 74 ena 121 
Large 97 185 83 47 5-4 97 
ARAR GTS Small 96 182 65 60 4.1 109 
Large 98 186 78 50 ee 104 
Mean,.... Small | 115.90 | 167.43 | 96.67 | 63.38 5-19 | 118.81 
Mean..... Large | 116.43 | 167.14 | 96.71 61.86 5.36 118.00 
Sigma..... Small 10.53 7.26 | 17.09 | 17.15 -78 14.06 
Sigma..... Large 10.62 9.20 | 14.28 | 13.93 -63 8.83 


that an equal difference would occur outside of these limits 
were even for chronological age, reading ability, grammar, and 
spelling. The chances for the differences in I.Q. and composi- 
tion were 1.18 to 1. 
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A report of one experiment suggests a rather practicable 
technique for comparing teaching procedures without equating 
groups, where evidence is available concerning the relative 
abilities of the groups in question.” Thus, if an experimental 
group with a lower mental age, in spite of this handicap, equals 
or exceeds the performance of a control group, it speaks well 
for the experimental method of teaching, provided no other 
variable is present to explain this relatively good performance. 

A suggestive method for determining the equality of groups, 
when data are not available for pairing pupils, follows: 2° 


Ideally, one should have sufficient information about each individual 
taking part in a control experiment to match pairs on the constants 
of their growth curves. Then, the only factors causing variations 
would be chance factors. In the absence of such ideal knowledge, 
however, it seems to me the use of four criteria will materially 
increase the certainty of results. 


4. Give a series of tests separated by intervals sufficient to permit 
a real growth in all subjects. At least three such tests should be 
given, 

B. Match pairs on achievement in Test 2 and growth from Test 
2 to Test 3. 

C. Predict the scores that will be made on Test 4 in both groups. 
Deviations of actual scores from predictions measure the 
validity of the matching. Compare groups by distributions of 
individual growths. If predictions and scores do not agree, re- 
match the two groups, and again predict. 

D. As soon as the groups are proved equal by criterion C, intro- 
duce the experimental factor into the experimental group. Pre- 
dict growth of control group. 


If now the control group makes the predicted score, the difference 
between experimental and control scores may safely be ascribed to 
the experimental factor. The variation of the actual from the pre- 
dicted scores for the control group „becomes a measuring stick for 
determining how much dependence may be placed on the differences 
between the experimental and control groups. 


27E, O. Melby and Agnes Lien, “A Practicable Technique for Determining the Kela- 
tive Effectiveness of Different Methods of Teaching,” Journal of Educational Research, 
XIX (April, 1929), 255-64. 

28S. A. Courtis, “Criteria for Determining Equality of Groups,” School and Society 
“XXXV (June 25, 1932), 874-8. 
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Needless to say, I personally believe a fifth criterion essential. All 
scores should be made comparable by being expressed in maturation 
units before matching and tabulations. Under such conditions, scores 
would be truly comparable, intercorrelations would rise, and prophecy 
could be based upon the results with assurance. 


The full import of the preceding paragraph may be clear 
only to those who are acquainted with the interest of Courtis, 
in formulating a biologic law of growth, with corresponding 
curves of maturation.?° 

Tn certain studies the investigator does not actually manipu- 
late directly an experimental factor, after groups have been 
equated, but chooses the subjects from conditions of the present 
or past to involve reasonably controlled conditions and an ex- 
perimental factor. For example, the experimenter may pair 
certain pupils, who have already attended for a period of 
years segregated schools for Negroes, with other pupils who 
have attended mixed schools (for white and colored children). 
The problem is to determine the effect of this variable (the 
type of school attended) on the achievement and attitudes of 
the pupils who are paired in terms of a variety of measures.*° 
Obviously, the experimental factor has been operating over a 
period of years, long before the investigator decides to select 
equated groups and to measure the effect of the variable on 
the pupils. 

Numerous examples of the various types of controlled group 
experimentation and of more detailed procedures for securing 
equivalent groups are available in two sources. A good sum- 
mary statement of techniques for securing parallel groups, 


208, A. Courtis, “The Derivation of Norms,” Journal of Experimental Education, II 
(March, 1934), 237-42. 
, “The Prediction of Growth,” Journal of Educational Research, XXVI (March, 
1933), 481-92. s = ae 

30 Mary A. R. Crowley, “A Comparison of the Academic Achievement of Cincinnati 
‘Negroes in Segregated and in Mixed Schools.” Unpublished Doctor's thesis, Teachers Col- 
lege, University of Cincinnati, 1931. A summary may be found in Abstracts Graduate 
Theses in Education, Teachers College, University of Cincinnati, 1927-1931, pp. 64-97. 
Cincinnati, Ohio: University of Cincinnati, 1931. 

Inez B. Prosser, “Non-Academic Development of Negro Children in Mixed and in 
Segregated Schools.” Unpublished Doctor’s thesis, Teachers College, University of Cin- 
tinnati, rosc. 
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especially in terms of feasibility and practicability in a class- 
room situation, follows: 31 


No one of the techniques described may be called the best. The 
use of chance is the most feasible when one wishes to use very 
large groups. The use of ordinary school classes, whose degree of 
equivalence is later checked, is to be commended since it causes 
little change in the usual schoolroom conditions. That groups should 
be equivalent with respect to measures of general intelligence has 
become rather well accepted among research workers in education. 
More careful experimenters go even further and strive to secure 
groups which are identical on the basis of several criterion measures 
including intelligence test scores, Since the use of several criteria as 
bases makes selection difficult, the use of composite scores would 
seem an effective technique. The use of composite scores, however, 
implies weighing of criteria, and research is needed to determine ap- 
propriate weights. Until this has been done, it would seem best to 
pair pupils with respect to intelligence test scores and to check the 
equivalence of other criteria, such as chronological age, previous 
achievement in the field of experimentation, study habits, personality 
traits, and physical condition by comparison oy means and standard 
deviations of measures which have been made of them. In addition, 
the groups should be checked for the purpose of determining whether 
or not they are approximately alike with respect to sex and race. If 
measures of traits other than intelligence are thus checked, it is fre- 
quently possible to make adjustments toward more perfect equiva- 
lents, or if adjustment is not possible, to make appropriate allowances 
for imperfections in equivalence, thus discovered, in formulating con- 
clusions. Whatever technique of securing equivalence is used, and 
whatever traits are considered, they should be the ones most appro- 
Priate to the problem and to the conditions under which the experi- 
ment must be conducted. The degree of elaborateness must be balanced 
against the artificiality of conditions which it may create, the cost of 
tests, the time, the number of cases, and other factors. 


Although more detailed discussion of the interpretation and 
reliability of differences in gains (involving probable error) 
between experimental and control groups, as well as of other 
statistical procedures (averages or central tendency, dispersion 
or variability, correlation, etc.) used in experimental studies, 

81 W. S. Monroe and M. D. Engelhart, op. cit. 


M. D. Engelhart, “Techniques Used in Securing Equivalent Groups,” Journal of Edu 
cational Research, XXII (September, 1930), 103-9. 
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is reserved for Chapter XI, attention at this time should be 
directed to the importance of the first problem, namely, the 
significance of differences in gains.** 


The preceding discussions of the significance of differences in gains 
should make it clear that caution must be exercised in interpreting a 
small difference and that the interpretation cannot be accomplished by 
the application of any formula or group of formulae. In general, it is 
necessary to inquire carefully and critically into the conditions of 
the experiment; then the best that can be done is to estimate the 
allowance that should be made for imperfections in the data. Since 
an estimate must be considered only an approximation, it follows that 
the interpretation of a relatively small difference in gains must be 
somewhat uncertain. When the difference is relatively large, definite 
conclusions may be justified, but even in this case they must be 
restricted to the conditions of the experiment. 


Laboratory experimentation. Group experimentation, as 
described above, is ordinarily conducted in classrooms for the 
purpose of evaluating various methods of instruction and has 
distinct value in aiding teachers to choose between instruc- 
tional procedures of varying effectiveness. However, it has been 
emphasized that this type of classroom investigation is neces- 
sarily more limited in its approach than the experimental 
analysis of learning and its underlying abilities. The latter 
type of experimentation is commonly conducted in the labora- 
tory where conditions may be carefully controlled, suitable 
apparatus and equipment provided, and subjects studied in- 
dividually.** Frequently, appropriate equipment must be de- 
vised to meet a given need. 

Attention is directed to the fact that relatively few graduate 
departments of education have made genuine contributions to 
educational psychology and experimental education through 
the medium of analytical studies of learning in the labora- 
tory: *4 


32 W. S. Monroe and M. D. Engelhart, op. cit, p. 75. 

28 F. N. Freeman, “The Place of Laboratory Experiment in Educational Research,” 
Review of Educational Research, 1V (February, 1934), 97-107- 

34G. T. Buswell, “The Laboratory Method in Educational Psychology,” Elementary 
School Journal, XXXII (May, 1932), 656-65- 
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The laboratory attack on educational problems is not new. It had 
its beginning in the laboratory which Wundt set up in Leipzig in 
1878, but for some reason it has received much less attention in 
institutions for the training of teachers than have other approaches to 
the problems of education. The number of departments of education 
which are contributing seriously to the laboratory studies in educa- 
tional psychology is notably small. To be sure, laboratory research 
is expensive, it is tedious, it of necessity must be limited to small 
numbers of cases. Nevertheless, it makes possible a type of minute 
analysis which no other method of study has provided. The primary 
function of the laboratory is to supply an analysis of the processes 
which must be carried on by the child in learning. The laboratory 
should supply to the teacher a method of understanding the mental 
processes of the child. Teaching is always intimately related to analysis. 
The teacher must give to the child an analysis of the processes to be 
learned which no beginner can supply for himself. It is only after a 
process has been learned that the learner is able to analyze it. 

A more complete characterization of the laboratory method 
and its procedures, as compared with group experimentation, 
follows: (1) provides the chief basis for theorizing, for ex- 
ample, Gestalt psychology; (2) tends to deal with the simpler, 
more fundamental processes, as in the case of the examples 
summarized at some length on subsequent pages of this chap- 
ter; (3) may not be directly concerned with immediately 
practical problems and may be motivated only by the type of 
intellectual curiosity which is found in “pure” chemistry or 
physics; (4) may cross the boundaries of other sciences such 
as neurology, biology, and physiology; (5) utilizes a limited 
number of cases, not only because of practicability, but due to 
interest in the patterns of behavior, mechanisms of learning, 
and maturation, problems for which individual analysis and 
study are the only practicable approaches; (6) may establish 
conditions of an atypical nature in attempting to control vari- 
ables rigidly, that is, in trying to free the subject of one set 
of interfering stimuli, he may be exposed to another set of 
conditions which makes both the situation involved and the 
attendant or resulting behavior unnatural (good examples of 
how the experimenter guarded against such a danger are found 
in the first laboratory study summarized on subsequent 
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pages); (7) is grounded in the belief that there are constants, 
universal principles, and uniform sequences in human nature, 
and that these transcend the generalizations which can be 
stated only in terms of statistical reliabilities (parallel-group 
experiments lead merely to statements of probability). This 
point of view returns to the first statement, namely, that 
laboratory research leads to theories, principles, and generali- 
zations of large import, especially in such fields as the psy- 
chology of learning. Investigators who use large numbers in 
group-testing studies well may cultivate the analytical skills 
and insights of the laboratory worker, while the laboratory 
technician in turn may visualize to advantage some of the more 
practical and pressing problems in the field.’ 

A discussion of the place of the psychological laboratory in 
educational diagnosis includes an analysis of different types 
of laboratory diagnosis, which are given below.** (Principles 
and illustrations of educational diagnosis are presented in the 
next chapter.) 


Laboratory studies of school activities involving skill 
Diagnosis through the aid of time records 

Diagnosis made through the photographing of eye-movements 
Diagnosis aided by motion and sound films 

Diagnosis by the aid of voice records 

Diagnosis of emotional tension 

Miscellaneous diagnostic symptoms studied in the laboratory 


pombe en 


It seems desirable to summarize in some detail the technique 
of a laboratory study of the experimental type. An excellent il- 
lustration of educational experimentation of the laboratory type 
is the work in arithmetic, reading, and other school subjects 
at the University of Chicago. These experiments have utilized 
extensively the technique of photographing eye-movements, 
and in some respects represent a planned attack on a number 


35 The authors are indebted to Gordon Hendrickson of the University of Cincinnati 
for this apt characterization of laboratory procedure. 

36G, T. Buswell, “The Place of the Psychological Laboratory in Educational Diag- 
nosis,” Educational Diagnosis, Chapter IX, Thirty-fourth Yearbook of the National So- 
ciety for the Study of Education. Bloomington, Il.: Public School Publishing Co., 1935. 
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of school-subject areas through analytical laboratory methods 
and psychological diagnoses. For these reasons it seems advan- 
tageous to draw the detailed illustration from this laboratory 
rather than to use briefer scattered examples from a number of 
institutions. The research programs of certain other labora- 
tories and universities, with special centers of interest, are 
represented in the bibliography of the present chapter and in 
other chapters dealing with normative-survey, genetic, causal- 
comparative, case, and correlation methods. 

The extended illustration used represents the first experi- 
mental study of eye-movements in looking at pictures, a very 
complex problem involving the use of elaborate equipment. 
The laboratory technique employed, that of photographing 
eye-movements, is an excellent example of the gradual de- 
velopment of laboratory procedure and equipment, to which a 
number of experimenters have contributed. It should be em- 
phasized that the only function of any laboratory instrument, 
however elaborate, is to refine the process of observation 
through control of conditions or through accuracy or per- 
manency of recording. Data are of value not because they are 
secured by means of laboratory apparatus, but in terms of 
whether they provide valid evidence for the solution of 
the problem at hand. Therefore, the quality of scientific study 
is not in the instrument itself but in the way the apparatus 
and the resulting data are used to accomplish a desired end. 
Use of equipment, such as provides silent or sound films, or 
eye-movement photographs, for purposes of observation only, 
may be considered normative-survey in character, or in certain 
instances genetic (if the emphasis is on norms of growth and 
‘development at various stages), and is illustrated in the appro- 
priate chapters (VIII and X). It is true that phases of a 
number of the eye-movement studies are of this observational- 
survey character, although the parts of the investigation 
reported in the succeeding paragraphs are definitely experi- 
mental in character, since the purpose and procedure were to 
note the effect on the perception of the subject as the result 
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of manipulating such experimental factors or variables as: 
presence and absence of color in pictures, silhouette and out- 
line forms, partly finished and complete pictures, and various 
types of directions in looking at pictures.*’ 


The method of securing a record of eye-movements consists in 
photographing a beam of light from a six-volt ribbon-filament lamp 
reflected first to the cornea of the eye from silvered glass mirrors 
and then from the cornea to a second set of mirrors, through a camera 
lens and a set of wedge prisms to a moving kinetoscope film. By 
means of the prisms back of each lens the beam of light from the 
eye is split into two beams, one of which is directed to a horizon- 
tally moving film and the other to a vertically moving film. In 
this way the movements of the same eye are recorded on both films. 
The direction of the pencil of light reflected from the cornea is 
changed with each movement of the eye. As the subject looks at a 
picture, a photograph is made which records the movements of the 
eye in a sharply focused line upon the two films. By means of a 
fan blade driven by a synchronous motor, the beam of light is inter- 
rupted thirty times per second between the lens and the film in such 
a manner that on the film the line of light appears as a series of dots, 
each dot representing one-thirtieth of a second of time. By counting 
the number of dots in each fixation pause it is possible to determine 
with precision exactly how long a person looked at each position in 
the picture. Only subjects were used whose vision was sufficiently 
normal to look at the picture without the use of spectacles. 

The apparatus was built for the particular purpose of this experi- 
ment in the workshop of the laboratories in educational psychology 
of the University of Chicago. Basically the apparatus is a large 
camera built in such a way that the two films can be moved con- 
tinuously during the process of photographing. The various lenses 
and mirrors are simply for the purpose of bringing to a focus on 
the film the reflection of the tiny spot of light from the cornea of the 
eye. The light which reflects on the eye originates under the table. 
It is passed forward through a series of lenses and then upward through 
two holes in the table, after which it strikes two circular mirrors and 
is reflected to the subject’s eyes. Instead of facing the camera lens, 
as has been necessary with previous pieces of apparatus of this sort, 
the subject is placed at right angles to the camera which gives him 
an open field of vision of whatever size is needed. Small pictures 
can be placed as close as the normal reading distance of twelve 
inches, whereas larger pictures can be set back whatever number of 


37G. T. Buswell, How People Look at Pictures: A Study of the Psychology of Per- 
ception in Art, pp. 10-6, Chicago: University of Chicago Press, 1935. 
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feet seems desirable. The provision of this larger field of vision ‘adds 
a great deal of flexibility to the uses of the apparatus. 

The subject sits in a specially constructed chair which can be 
raised or lowered to the proper height. When he is in position, the 
head-rest is lowered around the back of the subject’s head in order 
to eliminate head movements. On the whole, the position is com- 
fortable and, after a few seconds of adjustment, the apparatus causes 
the subject no inconvenience. A record of head movements is obtained 
for every picture by securing through a second lens on the apparatus a 
photograph of a beam of light reflected from a chromium bead on a 
pair of spectacle frames which the subject wears. The use of a head 
line, particularly for the vertical record, is absolutely necessary to 
insure precision in plotting the record. 

The films are plotted by projecting them through a stereopticon 
lantern. The vertical and horizontal position of each fixation is recorded 
on a numerical scale, and these positions are then codrdinated and 
marked on the picture. Points of reference for fitting the film record 
to the exact size of the picture were secured from initial fixations 
on four dots placed adjacent to each margin of the picture. For the 
complete investigation some 18,000 feet of film were used. 

A fixation of the eyes, of course, covers an area rather than simply 
a point. There is no means of knowing exactly how large the area 
of clear fixation is. The dots showing the position of fixations should 
be interpreted as the central points of areas of clear recognition which 
shade off gradually into areas of peripheral vision. 

In looking at a picture, just as in the process of reading, the eye 
moves in a series of quick jerks and pauses. The eye does not slide 
over the picture, as many people seem to think it does. The duration 
of the fixation pauses varies a good deal, a pause of 3 thirtieths of a 
second being very brief, one of 8 to ro thirtieths being quite 
common, and pauses of more than 20 thirtieths of a second occurring 
only in approximately 5 per cent of the cases. These movements and 
pauses of the eyes may be noted by direct observation, but the 
movements are too rapid to be counted accurately and the pauses 
too brief to be timed without the use of special apparatus. 

Photographic records of eye movements were obtained from two 
hundred individuals for the present investigation. Of this number 
twelve were elementary-grade children, forty-four were high-school 
pupils, and 144 were adult subjects. Of the adult subjects forty-seven 
were secured from the Art School of the Art Institute of Chicago 
and were persons who had from two to five years of special training 
in the field of art. Fourteen other subjects had made sufficient study 
of art to be classified as art students. The great majority of the re- 
maining adult subjects were college or graduate students. 

In the entire investigation fifty-five different pictures were used. 
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For one group of nine subjects photographic records were made from 
thirty-nine different pictures for each subject. Thirty-four subjects 
looked at twenty-one or more pictures each. For the remaining subjects 
fewer pictures were taken. However, the total number of completed 
records used in this report is 1,877. The only records discarded were 
those where the amount of head movement was so great as to render 
doubtful the plotting of the picture or where some ocular defect 
made the film record unreliable. 


In view of the fact that some educational workers have 
failed to recognize the part played by eye-movement photog. 
raphy in laboratory experimentation, it should be emphasized 
again that this equipment is a partial means of controlling the 
conditions of the experiment carefully and of recording be- 
havior accurately. Of course, an experiment must involve an 
experimental or variable factor which is manipulated, in addi- 
tion to control of other essential conditions or factors, and it 
is this experimental factor which may not be present or may 
not be recognized in certain of the laboratory investigations. 
Because of this difficulty, it seems desirable to point out the 
experimental variable in certain other laboratory studies where 
the typical procedure was to present to the same subject in 
succession varying types of materials, or situations, or direc- 
tions, and to note the resulting behavior. The same procedure 
was repeated with as many subjects as desired. 

The relationship between observational-survey and experi- 
mental work is illustrated in a group of diagnostic studies in 
arithmetic where the eye-movements of the subjects were 
photographed in doing different types of examples in column 
addition.** By means of a dictaphone a time analysis of the 
steps and of the behavior of the subject was also made for 
each of the four fundamental operations in arithmetic. With 
the difficulties of pupils thus observed and diagnosed, remedial 
teaching (an experimental factor) was done, and the gain 
noted and compared with the expected gain of a normal group 


38G. T. Buswell and Lenore John, Diagnostic Studies in Arithmetic, pp. 4, 14-6, 
47-8, 83. Supplementary Educational Monographs, No. 30. Chicago: Department of Edu- 
cation, University of Chicago, 1926. 
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of pupils for a similar period of time. Of course, diagnostic 
case studies are not typically experimental and are reserved 
for discussion in Chapter X. 

In another study, a psychological analysis of counting, the 
subjects counted stimuli (an experimental variable) in the 
forms of series of sounds, or flashes of light, or tactual impres- 
sions varying in rate and number from time to time.*® In a 
training experiment, simultaneous sounds and flashes of light 
were presented, and the effect of this training (an experimental 
factor) noted in terms of increased ability to count flashes of 
light. 

Yet another investigation, a study of the eye-movements in 
the various types of silent reading, included a series of experi- 
mental factors: (1) an increasingly difficult series of passages, 
(2) varying directions which changed the attention of the 
reader, (3) different types of analytical study, and (4) various 
types of reading material (English, English and foreign lan- 
guage mixed, and foreign language).+° 

Other illustrations of laboratory studies, psychological 
analyses, and classroom experiments. Since space does not 
permit additional detailed summaries of other laboratory 
studies, it must suffice to indicate where such information may 
be found. A year-book on educational diagnosis includes ex- 
tended discussions of the way in which laboratory and experi- 
mental procedures have contributed to educational adjustment 
in the various school subjects and activities, and of how physi- 
cal, intellectual, pedagogical, emotional, social, and environ- 
mental factors are associated with learning difficulty. In the 
present discussion of experimental work, the chapters (and 
individual contributions) dealing with arithmetic by Brueck- 


89C. H. Judd, Psychological Analysis of the Fundamentals of Arithmetic. Supple- 
mentary Educational Monographs, No. 32. Chicago: Department of Education, University 
of Chicago, 1927. Pp. x + 122. Reviewed by E. R. Breslich, School Review, XXXV 
(June, 1927), 467-8. 

40C. H. Judd and G. T. Buswell, Silent Reading: A Study of the Various Types. 
Supplementary Educational Monographs, No. 23. Chicago: Department of Education, Uni- 
versity of Chicago, 1922. Pp. xiv +160. Reviewed by W. F. Dearborn and D. A. 
Prescott. Elementary School Journal, XXII (May, 1923), 709-11- 
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ner and with music by Seashore are of especial significance.‘ 
Reference may also be made to a series of experiments con- 
ducted at Columbia University.** Obviously, it is impossible to 
call attention to all of the significant experiments which have 
been conducted during the past decade or two at various 
institutions. 

It will be recognized that the discussion of experimental 
research presented in this chapter is of necessity limited to a 
brief outline of the procedures involved. The investigator who 
would become proficient in experimental and laboratory tech- 
niques needs as a background detailed study of appropriate 
experiments such as were summarized above or are represented 
in the chapter bibliography under the names of: Buswell, 
Cattell, Charters, Freeman, Garrett, Gesell, Goodenough and 
Anderson, Hartshorne and May, Hudelson, Murchison, Mon- 
roe and Engelhart, Murphy, Schwesinger, Skinner, Starch, 
Thomas, Thorndike, Valentine, and Wood and Freeman. In 
general, these references are summaries of large numbers of 
selected experiments in a variety of fields rather than reports 
of a single study, and as such serve as guides to the reader who 
may turn from the summary to the full report of the original 
experiment as he desires. 

In the section of the appendix dealing with needed research 
are lists of problems which have been recommended for experi- 
mental investigation, especially under the topic of educational 
psychology and measurement. 

Evaluation of, and outlook for, the experimental method. 
It is probable that workers in education have been much too 
optimistic in their expectations that mosi of the major prob- 
lems of teaching and learning are being solved by application 
of the experimental method. Too many so-called experiments, 


41L. J. Brueckner and Others, Educational Diagnosis, Chapters XIV, XXI. Thirty- 
fourth Yearbook of the National Society for the Study of Education. Bloomington, IIl.: 


Public School Publishing Co., 1935. k 
42 E. L. Thorndike and Others, The Fundamentals of Learning. New York: Bureau of 


Publications, Teachers College, Columbia University, 1932. Pp. xviii + 638. Also see 
C. L. Hull, “Special Review of Thorndike’s Fundamentals of Learning,” Psychological 


Bulletin, XXXII (December, 1935), 807-23- 
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in equating groups and in interpreting results, have neglectea 
the various factors described in the preceding discussion. Pos- 
sibly quantity production has been emphasized to the partial 
neglect of quality.** 


We teach our students to be scornful of tradition and mere observa- 
tion and insist that all things must be subjected to the test of controlled 
experimentation. 

This is undoubtedly a healthy attitude to take if education is to 
become a science but the constant reader of the present-day educational 
literature cannot in his critical moments help but be troubled by the 
imperfections and ambiguities of our measurements and the inconclu- 
siveness of our sporadic experiments. When, for example, on such 
an important problem for educational theory and practice as the effect 
of equal practice on individual differences, whether equal practice 
increases or decreases them, we find out of twenty-four experimental 
studies twelve of them leading at least tentatively to the conclusion 
that differences are increased and twelve to the conclusion that differ- 
ences are decreased, we cannot help wondering about our experiments 
and about the conclusions derived from them. 


In view of the numerous problems of control, measurement, 
and interpretation in experimental research, the outlook for 
this method as a source of accurate generalization is not 
wholly optimistic, although its stimulating effect on teachers 
and pupils is significant. Monroe and Engelhart, after discuss- 
ing the crucial difficulties of definition of the experimental 
factor and adjustment of other educative factors to it, control 
of non-experimental factors, measurement of achievement, and 
generalizing, characterize the outlook for experimental re- 
search in education as follows.** 


When we consider the crucial difficulties encountered in experimental 
investigations, it is difficult to be very optimistic in regard to the 
improvement of research procedures so that the findings will be highly 
dependable. As we have indicated, there is evidence that experimental 
techniques are being improved, and it is possible to present a strong 
case in support of the statement that we are leaving the plateau 
period. It is more difficult to predict the future, but it seems doubtful 

43 V. A. C. Henmon, “Measurement and Experimentation in Educational Methods,” 
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whether we are justified in expecting that in time it will be possible 
to set up an experiment or a group of experiments that will yield 
definite and final answers to any question concerning the relative merits 
of a given educative factor. Some questions can be answered satis- 
factorily. A few have been answered. But for many questions, per- 
haps most questions, it is likely that we are not justified in expecting 
more than an “indication.” 

Controlled experimentation, however, is worth while. In addition 
to the dependable information that may be contributed, there are 
valuable by-products. Experimental investigations are stimulative. Ex- 
periments with the project method have stimulated a greater interest 
in this instructional procedure, and we know that under certain con- 
ditions it works. 


An amplification of this point of view, based on an examina- 
tion of a large number of experimental studies after the fore- 
going statement was written, reaches essentially the same 
conclusion.** 


It is hazardous to predict the future, but a critical examination 
of a large number of experimental studies does not induce one to 
give an optimistic answer. In fact it seems appropriate to ask if 
there are any best methods of teaching. It appears reasonable that 
the effectiveness of a method may be conditioned to such an extent 
by a teacher’s confidence in it and the zeal and skill with which 
she applies it that the method itself is a minor factor in teaching 
success, Furthermore, mechanized instruction does not represent our 
ideas of good teaching. We say that the superior teacher is resourceful 
and constantly adapts her instruction to the needs of her pupils as 
they are revealed from day to day. A method that is best to-day may 
not be best next week. A method that is best for one class may not 
be best for another. These hypotheses, which are supported by con- 
siderable experimental evidence, are not compatible with a list of 
methods ranked in order of merit. In other words it appears doubtful 
if the relative merit of comparable methods of teaching is a stable 
thing and a determination may be applicable only to a particular 
teacher and a particular teaching situation. 

This conclusion raises the question of the value of experimental 
studies of methods of teaching. If we cannot expect to determine the 
relative merits of methods, what is the value of such research? 
Experimental inquiry during recent years has added to our under- 
standing of the teaching process. A number of experiments have con- 


45 W. S. Monroe, “Controlled Experimentation as a Means of Evaluating Methods of 
Teaching,” Review of Educational Research. IV (February, 1934), 36-42. 
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tributed to our understanding of the learning process. Considerable 
progress has been made toward identifying the factors that affect 
learning. We have learned something of the conditions under which 
certain methods are effective. In general, it may be said that experi- 
mental studies have contributed to more precise and critical thinking 
about methods of teaching. Experimental inquiry is likely to be 
stimulating to the teachers participating in it and hence may be 
valuable as a supervisory procedure. Hence, a strong argument may 
be advanced in support of experimental studies of methods of teaching. 

It may be observed, however, that since the crucial difficulties of 
experimental research have been revealed, an investigator should not 
expect to receive recognition for his work unless he employs the best 
techniques for dealing with these difficulties. The experimental factor 
should be defined and restricted so that the findings may be inter- 
preted with precision. The nature and scope of the pupil achievement 
to be measured should be specified. Attention should be given to 
the control of non-experimental factors, especially those relating to the 
teacher. Measuring instruments adapted to the requirements of the 
problem should be selected or constructed. The experiment should 
be continued for a period of time sufficient to reveal the effect of an 
extended application of the method. Finally, the experimenter should 
consider the dependability of the obtained difference with reference 
to the control of non-experimental factors and the possible presence of 
systematic errors of validity. 


Douglass, after pointing out that the controlled experiment 
has not fulfilled the expectations of its proponents of a decade 
ago, discusses difficulties to be met in this type of investiga- 
tion; *° 


1. The equating of pupils in small experimental groups cannot be 
done with any certainty of accuracy; therefore, where possible, hun- 
dreds rather than dozens of pupils should be used. 

2. The teacher or teachers employing two or more methods to be 
studied should be equally skilful in the use of each. 

3- Measurement of growth during the experimental period has 
stressed written subject-matter tests of a highly factual and detailed 
nature to the virtual neglect of concepts, ideals, attitudes, tastes, 
and general powers or skills. 

4. Whatever units of measurement are employed, whether reduced 
to standard scores or scaled units, represent an approximation which 
renders any further statistical treatment useful only for obtaining good 


46H. R. Douglass, ‘Scientific Investigation of Instructional Problems,” Journal o) 
Educational Research, XXIX (October, 1935), 130-8. 
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estimates better than non-technical methods but not mathematically 
accurate. 

5. The statistical technique necessary is often more complicated 
than any person, except the best trained research workers, is able to 
employ. 

6. Interpretation of coefficients of correlation, probable errors of 
estimate, and other measurements of the efficiency of prediction is 
more complicated than most research workers and many statisticians 
think. j 

7. One may not assume confidently that the method apparently 
more effective for one teacher or group of teachers will be more 
effective for all teachers, or even for the majority of teachers or 
the same teachers, under conditions different from the experimental 
situation. 


Concluding statement. This chapter has defined the ideal 
conditions for use of the experimental method in terms of the 
law of the single variable, although recognizing that rarely 
in dealing with human beings can all factors be held constant 
except the experimental variable. The problems involved in 
classroom and group experimentation have been considered at 
some length, with a frank admission of the real difficulties 
present in such investigations. A rather detailed illustration of 
analytical laboratory studies has been presented, as promising 
a solution to educational problems not susceptible of ready 
attack by group-testing methods. 

Though recognizing the justice of the foregoing criticisms of 
past experimentation in education and the difficulties involved 
in applying experimental and laboratory methods to any social 
science field, education and psychology included, the writers 
wish to close this chapter with a note of optimism by a more 
extended statement of the stimulating effect experimentation 
may have on the educational worker.“ This point of view is 
presented with full recognition of the fact that in all proba- 
bility few classroom teachers, supervisors, or administrators 
will conduct refined and crucial experiments which provide 
valid evaluations of educational procedure. 


47B. R. Buckingham, “The Public-school Teacher as a Research Worker,” Journal of 
Educational Research, XI (April, 1925), 235-43- 


526 EDUCATIONAL RESEARCH 


Finally, their [experiments’] reaction upon the teacher spiritually 
must not be left out of account. The fresh point of view which 
research always engenders should not be denied to the teacher. The 
spirit of inquiry, of open-minded alertness to the problems which 
arise in teaching, will make the teacher free. It will lead him to seek 
problems and do something, even though it may be but a little, in 
their solution. Moreover, it will make him expert as a teacher and 
will make his calling more attractive. Indeed, when looked at from 
this point of view the teacher's occupation becomes fascinating. He 
has children to study—not stones, bugs, fossils or old manuscripts, 
but the most interesting of all possible materials—namely, human 
beings. Moreover, he has at hand human beings at their most engaging 
period—childhood and youth. And his children never grow old. In 
constant procession they present to him, always at the level of child- 
hood, their innumerable interesting aspects. Yet each is different 
from the other—different in strength, talent, and character; different 
in origin, growth, and need. If teaching these children is to include 
studying them, the job of teaching takes on new meaning. Its scope 
is broadened. Its meaning is enriched. No other calling may then be 
compared with it. It is the great adventure. 


PROBLEMS AND EXERCISES 


1. Select a completed experimental study, and characterize and evalu- 
ate it in terms of the principles of research discussed in this 
chapter, for example: 

(a) Laboratory or group classroom experiment 

(b) Control of variable factors 

(c) Isolation of the experimental variable 

(d) Adequacy of measurement procedures 

(e) Interpretation and significance of results 

Choose an experimental problem which might well be made, and 
outline a plan for its solution, applying so far as possible the 
principles of experimental research developed in this chapter. 


n 
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CHAPTER X 


OTHER METHODS OF RESEARCH—ESPECIALLY ADAPTED 
TO THE ANALYSIS OF COMPLEX CAUSAL RELATIONSHIPS 


Introduction. In the immediately preceding chapters, three 
methods of educational research have been discussed, namely, 
the historical, the normative-survey, and the experimental 
methods. In addition to’ these three well-known types, there 
are a number of other research procedures which have been 
developed : (x) the causal-comparative method, (2) the correla- 
tion method, (3) the case-study method, and (4) the genetic 
method. 

These methods of investigation have developed in response to 
the need for means of research suited to the study of the com- 
plex phenomena of biology, economics, sociology, psychology, 
and education. They represent in each instance an important 
attempt to supplement the older and more widely known types 
of research by offering new modes of procedure, or new com- 
binations of old procedures, adapted to the study of the many 
new kinds of problems which these relatively modern sciences 
have introduced. These newer methods of research represent 
important developments in the study of complex problems of 
education. In fact, they may offer more promise for penetrating 
research than some of the older methods. 

Although the four procedures discussed in this chapter do 
not introduce any new means of gathering data, they do involve 
new purposes in the collection of data, with new patterns of 
treatment. While each method shares certain of its procedures 
with other methods of research, each contributes a distinctive 
approach to the study of the problems of education. The causal- 
comparative method, which should be more widely known, will 
be discussed first. 
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THE CAUSAL-COMPARATIVE METHOD 


Definition of the causal-comparative method. The causal- 
comparative method of research seeks to establish causal rela- 
tionships by comparing the circumstances associated with 
observed effects and by noting the factors present in those 
instances in which a given effect occurs or does not occur. 
The method may stop with a study of the likenesses and 
differences in the conditions attending either the occurrence or 
non-occurrence of some phenomenon or may be extended to a 
comparison of the circumstances surrounding both the instances 
in which the phenomenon does and does not occur. Typically 
it does not go as far as the correlation method which associates 
a given amount of change in the contributing factors with a 
given amount of change in the consequences, however large or 
small the effect. The method always starts with observed effects 
and seeks to discover the antecedents of these effects. 

The causal-comparative method differs from historical re- 
search (which may also be directed towards the establishing of 
causal relationships), in that it deals with current situations. 
It differs from normative-survey research in that the latter is 
concerned chiefly with status, rather than with the antecedents 
of educational effects. It differs from experimental procedure 
in that the latter carefully controls its variable factors, whereas 
the causal-comparative method makes observations under nor- 
mal or field conditions. One might, in fact, be justified in calling 
the present procedure the “uncontrolled experimental” method, 
except that, according to customary usage, the term experi-* 
mental implies controlled rather than uncontrolled conditions. 

The causal-comparative method is admirably adapted to 
many types of field studies seeking to establish causal connec- 
tions where, for some reason or other, experimentation in the 
conventional sense is not feasible. This may be the case because 
of the impracticability of performing extensive social experi- 
ments or of holding all of the important factors constant. For 
example, problems of school support, character development, 
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the effectiveness of health education, and the like do not lend 
themselves to rigorous experimentation. A second type of situa- 
tion, to which experimental procedures are not readily applicable, 
is the study of the interrelationships of factors as they exist 
under natural conditions, in contrast to the artificially con- 
trolled conditions entailed by experimentation. As previously 
emphasized, any distortion of natural conditions may intro- 
duce, or eradicate, factors and relationships which significantly 
alter one’s findings. A third situation for which experimen- 
tation is not adapted arises when one is more interested in 
discovering the antecedents of a particular case than in dis- 
covering or verifying a large generalization concerning such 
antecedents. This may be the situation, for example, in study- 
ing the factors contributing to a high cost of education in 
some locality, or in ascertaining why unit costs of two school 
systems differ. 

Illustrations of causal-comparative research, Many ap- 
plications of this method are recorded in educational literature. 
For example, Barr * used this procedure in studying character- 
istic differences in the teaching of effective and ineffective 
social-studies teachers. One purpose of the investigation was 
to discover why some teachers succeed and others fail. The 
investigator began the study by selecting, on the basis of 
carefully chosen criteria, superior and inferior teachers. He 
then observed carefully and systematically a large number of 
characteristics of these different teachers, to determine what 
elements entered into and rendered the teaching good or poor, 
Finally, the investigator compared the data that he had col- 
lected relative to each teacher to discover elements common to 
effective and ineffective instruction. From such a comparison it 
was possible to list characteristic differences between the teach- 
ing performance of good and poor teachers, at least in the 
specific field of instruction under investigation, 


1A. S. Barr, Characteristic Differences in the Teaching Performance of Good and 
Poor Teachers of the Social Studies, Bloomington, Ill.: Public School Publishing Co., 
1929. Pp. viii + 127. 
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In an earlier application of this research method, Freeman * 
studied the motor factors which contributed to excellence in 
penmanship. Using motion-picture equipment, he photo- 
graphed the arm and finger movements of various persons, rep- 
resenting a writing expert, a good adult writer, two poor adult 
writers, and children. From an examination of these pictures, 
Freeman was able to identify movements made by good and 
poor writers, and thus to describe the movements that tended 
to cause one to become a good or a poor writer. 

Freeman supplemented this research by subjecting his find- 
ings to an experimental test in terms of their effectiveness as 
teaching aids. As suggested by the preceding study, the recom- 
mendations were further tested with groups of children who 
made marked gains in speed and form as compared with groups 
taught hand-writing in the usual fashion. The insights gained 
from the causal-comparative study, therefore, proved effective 
in contributing to better teaching. This two-type study affords 
an interesting illustration of the use of one method to check 
upon the practical utility, and perhaps the validity, of the 
findings of another method. 

Baker * employed the causal-comparative method to study 
typical differences between bright and dull pupils. In the pref- 
ace to his report, he purposed “to go behind the scenes of 
testing and interpret, if possible, the differences in general 
intelligence in terms of the psychology of learning.” He an- 
alyzed the mental and physical activities of the pupils, com- 
paring the elements common to the group of bright pupils 
and those common to the group of dull pupils but absent from 
the activities of the bright pupils. This process enabled him to 
draw distinctions between the characteristic behavior of the 
two groups. He was particularly interested in the way children 
learned school subjects. 


2 Frank N. Freeman, The Handwriting Movement: A Study of the Motor Factors of 
Excellence in Penmanship. Chicago: University of Chicago Press, 1918, Pp. xvi + 169. 

3H. J. Baker, Characteristic Differences in Bright and Dull Pupils: An Interpretation 
of Mental Differences, with Special Reference to Teaching Procedures. Bloomington, Ill.: 
Public School Publishing Co., 1927. Pp. viii + 118. 
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Numerous other studies employing the causal-comparative 
method have been made. The reader may be interested in the 
following as further illustrations: Brown * investigated the 
unevenness of abilities in dull and bright children; Wilson * 
studied differences in their learning; Carroll è analyzed their 
generalizations. The social rating of the best and poorest 
high-school pupils was investigated by Sangren while 
Upshall *® and Pressey? ascertained differences between good 
and poor students at the college level. Scates+° examined 
the records of college students who had been dismissed for 
poor work and were subsequently readmitted, in an effort to 
identify elements characteristic of those who were finally 
successful and of those who were unsuccessful after their re- 
turn. Watson ™ studied the possible areas of difference between 
students for whom predictions of outstanding success or failure 
were made. Mead * made an investigation of the qualities of 
merit possessed by superior and inferior teachers. 

Other studies have been directed to the analysis of the char- 
acteristics of a single group only, such as a high group or a 
low group. For example, Flemming * investigated the signifi- 


4A. W, Brown, The Unevenness of Abilities in Dull and Bright Children. Teachers 
College Contributions to Education, No. 220. New York: Teachers College, Columbia 
University, 1926. Pp. vii + 112. 

5F. T. Wilson, Learning of Bright and Dull Children. Teachers College Contributions 
to Education, No. 292. New York: Teachers College, Columbia University, 1928, Pp. 56. 

6 Herbert A. Carroll, “Generalization of Bright and Dull Children: A Comparative 
Study with Special Reference to Spelling,” Journal of Educational Psychology, XXI 
(October, 1930), 489-99. 

7P. V. Sangren, “Social Rating of Best and Poorest High School Students,” Journal 
of Educational Psychology, XIV (April, 1923), 209-14. 

8C. C. Upshall and Harry V. Masters, “Differences Between Good and Poor Students 
Chosen on the Basis of College Entrance Test Scores,” Educational Administration and 
Supervision, XIX (October, 1933), 507-10. 

9L. C. Pressey, “What Are the Crucial Differences Between Good and Poor Students?” 
Research Adventures in University Teaching, pp. 4-10. Bloomington, Ill.: Public School 
Publishing Co., 1927. 

10 Douglas E. Scates, “Selective Admission and Selective Retention of College Students 
at the University of Chicago,” Chapter V, “A Study of Dismissed and Readmitted 
Students,” pp. 251-352. Doctor’s thesis, Chicago: University of Chicago, 1926. Pp, 383. 

11 Gladys H. Watson, An Exploration oj Some Possible Areas of Difference Between 
Students for Whom Predictions for Outstanding Success or Failure Are Made. New York: 
Teachers College, Columbia University, 1931. Pp. 99. 

12A. R. Mead, “Qualities of Merit in Good and Poor Teachers,” Journal of Educa- 
tional Research, XX (November, 1929), 239-59. 

18 E. G. Flemming, Predictive Value of Certain Tests of Emotional Stability as Ap- 
plied to College Freshmen. Archives of Psychology, No. 96. New York: Columbia Unj- 
versity, 1928. Pp. 61r. ` 


OTHER METHODS OF RESEARCH 537 


cance of certain mental, physical, and character traits for 
success in high school, by noting those traits that were ex- 
hibited by the successful high-school pupils. In similar fashion 
O’Brien ** examined the records of failing high-school pupils 
to find common characteristics; Reavis ** observed the factors 
contributing to poor attendance in rural schools. Van Den- 
burg +° studied the causes of elimination of pupils from the 
public schools in New York City, and Turney * analyzed 
factors other than intelligence which affect success in high 
school. 

While such studies as these may be regarded as falling under 
the causal-comparative method, they do not, however, follow 
the method through to a complete and rigorous application. 
That is, instead of examining the characteristics of the con- 
trasting groups to make certain that factors which the observer 
considers distinctive really do not occur in equal degree in the 
opposite group, these studies depend upon a comparison with 
what the observer regards as normal. The degree to which the 
conclusions are valid, therefore, depends upon the success with 
which the observer can judge what is normal and what is 
peculiar to a special group, after he has observed many charac- 
teristics. Studies of this type should be regarded as exploratory 
and suggestive rather than as final. 

Comparative education. It is interesting to note that the 
field of comparative education may be approached from 
the point of view of causal-comparative analysis. That is, in- 
stead of merely studying the ways in which the school systems 
of various nations differ, one may go further and set these 

14 Francis P, O'Brien, The High School Failures: A Study of the School Records of 
Pupils Failing in Academic or Commercial High School Subjects. Teachers College Con- 
tributions to Education, No, 102, New York: Teachers College, Columbia University, 
1919. Pp. vii + 97. 

15 George H. Reavis, Factors Controlling Attendance in Rural Schools, Teachers College 
Contributions to Education, No. 108. New York: Teachers College, Columbia University, 
ee Van Denburg, Causes of the Elimination of Students in Public Secondary 
Schools of New York City. Teachers College Contributions to Education, No, 47. New 
York: Teachers College, Columbia University, 1911. Pp. iv + 206. 


17 A. H. Turney, Factors Other Than Intelligence Which Affect Success in High School. 
Minneapolis: University of Minnesota Press, 1920. Pp. t25. 
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differences in relation to their antecedents which lie in the 
social and political philosophy of the people. Sandiford *® notes 
typical differences in the school systems of England and Ger- 
many and relates them to basic attitudes of the people. Thus 
German education is described as highly systematized and 
organized on the basis of a national administration, while 
the English system is relatively individualistic, local, and un- 
organized, with much overlapping. The thinking of the two 
nations recognizes different criteria and accepts different goals. 
Kandel "° further traces variations in educational organization 
and practice to the political, social, and cultural forces which 
also shape other aspects of national life. In approaching the 
study of comparative education in this way, one is making 
a study of current conditions by drawing upon contemporary 
and past factors that have contributed to these conditions. The 
causal-comparative method is thus closely related to the his- 
torical-comparative method; it is appropriate for the same 
type of analysis, but is concerned with the explanation of 
conditions observable at the present time rather than with 
conditions in the past. 
Mill’s canons of agreement and double agreement. The 
logic underlying the causal-comparative method was set forth 
in the nineteenth century by John Stuart Mill, in his first and 
third canons of logic. Mill called these canons the principle 
of agreement and the principle of double agreement. The first 
one may be stated as follows: “If two or more instances of the 
phenomenon under investigation have only one circumstance 
in common, that circumstance may be regarded as the probable 
cause (or effect) of the phenomenon.” 2° There are three 
aspects of this statement which are of considerable interest. 
First, it admits the complexity of causal relationships by in- 
dicating that one cannot determine simply from observing the 
18 Peter Sandiford and Others, Comparative Education: Studies of the Educational 
Systems of Six Modern Nations. London: J. M. Dent and Sons, Ltd., 1018. Pp. x -+ goo, 
ae F ees Comparative Education. Boston: Houghton Mifflin Co., 1933. Pp. 


20 F. W. Westaway, Scientific Method: Its Philosophical Basis and Its Modes of Appli- 
cation, pp. 203-5. (Third edition.) London: Blackie and Son, Ltd., 1924. 
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associated occurrence of various conditions (any one or a 
group of which may be regarded as “the phenomenon under 
investigation”), which factors are causing others, and which 
conditions are resulting from others. The direction of causation 
must be inferred from observation and analysis of other ele- 
ments of relationship than simply occurrence together. 

The second aspect of the statement to be commented upon 
is that it is put in idealized form. That is, it calls for “only one 
circumstance in common.” As pointed out in the preceding 
chapter in connection with experimentation, it is impossible in 
practical situations to have only one variable factor; there are 
countless factors contributing to every situation. All that can 
be expected is to control the most important factors. Similarly 
in the application of Mill’s canon of agreement, we cannot 
hope ever to find a single element (in addition to the “phe- 
nomenon under investigation”) which is constant from one 
situation to another. Such general factors as time, space, mass, ` 
reaction, energy, and many others pervade all situations, either 
in effective or potential form, and it is highly idealistic to 
speak of “only one” circumstance in common. What we do 
therefore, as in experimentation, is to give attention to those 
circumstances which appear most likely to be significant for 
our purposes, and to note their occurrence or absence. 

A third important aspect of the statement is that it is not 
made in absolute form; it says merely that the circumstance 
identified as present in various situations “may be regarded 
as the probable cause.” As a matter of fact, when used alone, 
this canon is not to be regarded as rigorous logic. It must be 
supplemented by the third canon, that of double agreement. 
For example, if one notes only those factors which are uni- 
versally present (within the range or scope of his observation), 
he may find that many are not true causes at all, but are imma- 
terial in differentiating between one phenomenon (result) and 
another. For example, if one is studying good teaching, he may 
find that all of the teachers he observes are women, that they 
teach during daylight hours, that they teach in a particular 
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state, etc. One can guard against unwarranted conclusions to 
a certain extent by extending his observations to situations 
that vary in as many ways as possible, but use must be made 
of the principle of double agreement in order to be more 
certain. 

Mill’s third canon extends and completes the logic of his 
first canon, so far as strict logic goes. This third canon 
points out that, in order to isolate factors which are more prob- 
ably causes, one must find those which are absent when the 
phenomenon (result) does not occur, That is, referring to the 
illustration in the preceding paragraph, one may discover that 
such factors as sex, time and place of teaching, etc., occur in 
instances of poor teaching in the same manner as in instances 
of good teaching. Therefore, they are not significant causes of 
good teaching, because they are not absent in all cases where 
good teaching does not occur. 

Mill’s canon of double agreement is stated as follows: 2% 


If two or more instances in which the phenomenon occurs have only 
one circumstance in common, while two or more instances (in the 
same department of investigation), in which it does not occur, have 
nothing in common save the absence of that circumstance, the cir- 
cumstance in which alone the two sets of instances differ is the 
effect, or the cause, or an indispensable part of the cause, of the 
phenomenon. 


As observed in connection with his first canon, the direction 
of causation cannot be determined simply from association, 
and again the statement is made in highly abstract, idealized 
_terms, for circumstances do not exist singly. This element of 
idealization is also introduced in the present canon in a nega- 
tive form, for it assumes situations in which the phenomenon 
(result) does not occur that “have nothing in common” with 
the situations where the phenomenon does occur, except “the 
absence” of some factor. For any practical purpose such ab- 
stractions must be modified to recognize the presence of many 
common elements in all situations, and the statement must be 


21 Ibid., pp. 207-2 
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interpreted as applying to factors which are of most signifi- 
cance. 

Applications to causal-comparative research. For the pur- 
pose of guiding practical research of the causal-comparative 
type, Mill’s canons may be summarized as stating that causes, 
or indispensable elements of causes, of a given observed result 
may be ascertained by noting elements which are invariably 
present when the result is present, and which are invari- 
ably absent when the result is absent. One may make the 
application of this principle concrete in his thinking by regard- 
ing good teaching as a “result,” or “phenomenon,” and poor 
teaching as the absence of the result or phenomenon (i.e., as 
the absence of good teaching). A high quality of legibility in 
specimens of handwriting may be regarded as the presence 
of a result or phenomenon (probably caused by the common 
circumstance of a certain combination of arm, wrist, and finger 
movement), while illegible instances of handwriting may be 
considered as the absence of the phenomenon (legibility) and 
correspondingly the absence of the causal factor (the given 
combination of arm, wrist, and finger movement). The various 
studies mentioned earlier in this chapter afford practical illus- 
trations of the application of this logic. 

In carrying out this type of research in full, there are three 
important steps in the logic to be met. First, one will gather 
data on factors (circumstances) present in cases where the 
given result occurs, discarding those elements which are not 
universally present. Second, one will do the same thing 
for cases where the given result does not occur. Third, one will 
compare his two sets of data and, in effect, subtract the second 
set from the first set. Or, if one is interested in causes of the 
result not occurring, he will eliminate from both of his lists 
those elements common to the two. If his observations have 
extended over a sufficiently wide range of circumstances, he 
may conclude that the factors remaining in his two lists are 
respectively causes of the result’s occurrence and of its non- 
occurrence- 
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In addition to these steps of logic, many others must be 
included in the administration of actual research. For example, 
if one is studying teaching, he must have some means of 
identifying good or poor teaching. The establishing of criteria 
for such identification is a step that often causes considerable 
difficulty. : 

Limitations of Mill’s canons and of causal-comparative 
research, The first limitation of this research method is that 
it is essentially dichotomous. Both results and causes are 
thought of as either occurring or not occurring, whereas in 
reality they both may occur in a great many degrees. That is, 
we do not have merely good teaching and poor teaching, but 
we have in the main average teaching (the average is usually 
the most prevalent condition), with the quality of teaching 
shading upward and downward from this average by very fine 
degrees. The causal-comparative method does not recognize 
these various degrees in the presence or absence of phenomena, 
but considers only those instances that are recognized as good 
or bad, poor or excellent, etc. All other degrees of excellence 
or performance are disregarded. Thus a dichotomy in the 
phenomenon (the result) is assumed by this method. 

Likewise a dichotomy is assumed for each of the factors or 
circumstances noted. The element is either present or absent. 
In actuality, factors are always present in some degree, and 
this degree may usually vary from zero to a very strong sig- 
nificance by fine degrees. The full recognition of degrees of 
existence in both results and contributing factors is achieved 
by the correlation method, which of course has its own charac- 
teristic limitations, as analyzed later in this chapter. There 
is one way in which causal-comparative research may approxi- 
mate degrees in the conditioning factors, and that is to note 
the number of instances in which they occur. That is, instead 
of assuming that a factor is either present in full force or else 
absent entirely, one assumes that the importance of a factor 
may be indicated roughly by the proportion of instances in 
which it is present. This type of reasoning does not require 
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that a factor be present in all cases where the result occurs, 
nor absent in all cases where the result does not occur, but 
rather recognizes that a factor may be a cause to a certain 
degree if it is present in more cases where the result occurs 
than where the result does not occur. This type of reasoning 
was not contemplated by Mill’s canons, but constitutes a useful 
present-day extension of his conception of this method of 
research and is supported by modern statistical methods. 

A second limitation grows immediately out of the first one. 
It is that the method, particularly when applied strictly, will 
not reveal all of the causes. If one notes only causes which 
are invariably present when the result occurs and invariably 
absent when the result does not occur, he is overlooking all of 
the causes which contribute in a moderate degree to the occur- 
rence of the result. This may be serious if one expects to make 
practical applications of his conclusions. When the element of 
frequency of occurrence and the concept of partial or contrib- 
uting causes are introduced, the method becomes more sensi- 
tive and will reveal many more of the factors which enter in 
as causes. It does not, however, give any evidence of the inter- 
relation and interdependence of these causal factors. That is, 
to what extent is a given factor contributing in its own right to 
a result, and to what extent does this factor contribute simply 
because some other factor is acting upon it and making it 
behave in a certain way? The analysis of interdependence and 
uniqueness in the contribution of factors must be left to partial 
correlation methods (to be discussed shortly), in combination 
with a logical analysis of structural relationships. 

A third limitation of the causal-comparative method arises 
from the fact that it necessitates observation of many cases 
arising from a large variety of circumstances. If the variety is 
not sufficiently great, one may draw erroneous conclusions. 
This requirement of variety means that the method is not well 
adapted to the study of an individual case, such as discovering 
the reasons why a pupil cannot read well. For such purposes 
the diagnostic case-study procedure is appropriate; but diag- 
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nosis will depend upon insights and generalizations growing 
out of experimental, causal-comparative, normative-survey, 
and other types of investigation. Each research method is 
valuable for certain purposes and relatively worthless for 
others; an important part of understanding the value of any 
method lies in knowing its limitations so that it will not be 
used where ineffective or misleading. 

Good ideas and careful observation required. The causal- 
comparative method of research in its early stages is essentially 
one of analysis. Out of the totality of elements which are 
present in any situation, one has to select certain ones for 
observation. To think of all of them is next to an impossibility, 
for they are countless. Many elements are at present too subtle 
to be sensed directly and too fleeting to be identified: some of 
them may be significant, and others may be immaterial. In 
selecting factors (elements) to observe, one must use his best 
judgment. Factors which he excludes intentionally or uninten- 
tionally cannot enter into his conclusions, except in so far as 
they may be reflected in other factors that are partially influ- 
enced by them. The importance of an analytical comprehension 
of the field to be studied should be apparent. The construc- 
tion of one’s list of factors is closely related to the preparation 
of observational and other schedules, and the making of appro- 
priate categories, discussed in Chapters VII and VIII. It isa 
problem that requires insight, thought, and careful work. Upon 
this step depends much of the ultimate quality of the research. 

Having decided upon a list of factors to study (which, in 
practice, will only be completed after preliminary testing), 
attention shifts to the care with which these factors are to 
be observed. We mean to include here all forms of observation, 
in the scientific sense, embracing measurements of all kinds 
and mechanical or indirect observation as well as direct. One 
will use whatever technique is appropriate to the gathering 
of each kind of data which he desires to secure. 

Accurate and meticulous observation is particularly impor- 
tant in research which is seeking causes, since factors present 
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in small amounts may be of large significance. The most strik- 
ing illustrations can be drawn from the field of physiology. For 
example, vitamins are present in foods in such minute amounts 
that up until recent times they were not known to exist. Hor- 
mones are secreted into the blood stream by the endocrine 
glands in such minute quantities that they were overlooked 
until recently. Yet vitamins and hormones are so important in _ 
the everyday well-being of the individual that they may be 
ranked among the essential elements controlling physiological 
metabolism. The human body will react to some of these sub- 
stances in a dilution of one part in one million. At present it is 
not known whether there are factors of such minute propor- 
tions in one’s environment which affect him psychologically 
to the same extent that vitamins and hormones do physiologi- 
cally; the answer to such a question must await future devel- 
opments. Similar refinements in other fields do, however, serve 
to keep one alert to the possibility of discovering significant 
influences emanating from factors which are observable only 
in very small quantities. To detect the presence of minute 
elements which may be important, one must use measuring 
instruments that are correspondingly refined, and he must em- 
ploy a penetrating analysis that is equal to the problem to be 
solved. 

Care in interpretation of findings. The causal-comparative 
method is directed toward the discovery of antecedents or 
factors contributing to observed effects. It should be made 
emphatic that the inferring of causes is an extremely difficult 
and precarious matter, especially in social fields where rela- 
tionships are so complex that it is nearly impossible to deter- 
mine definite causes. Most persons will need to revise greatly 
their notion of causes, for they must not think of absolute 
causes but of factors which enter into a complex pattern of 
interrelationships so as to contribute their influence (much or 
little) toward or against the eventuation of a certain condition, 
Any result is the resolution, for that particular time, of the 
interplay of many diverse forces. Just as there are negative 
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forces, from the standpoint of a particular accomplishment, sca 
factors which enter in to keep a result from being “more so” 
must be regarded as causes, just as well as factors which act, 
for that particular occasion, in a positive way to keep the 
result from being “less than” it is. 

Factors present in all or most of the cases observed, regard- 
less of the occurrence of the given phenomenon (result), 
cannot necessarily be regarded as negative factors. They may 
be simply essential factors, for any degree of the result. 
Mathematicians have long distinguished between necessary 
conditions and sufficient conditions. Many conditions are 
necessary for conventional teaching of any kind, to refer again 
to the illustration that has been used, but these conditions 
which are common to all forms of conventional teaching are 
not sufficient to cause superior teaching. They may, neverthe- 
less, be regarded as causes, being essential for teaching of the 
usual type. On the other hand, some of them may actually 
be retarding influences, or negative causes, so far as good teach- 
ing is concerned. This will depend on one’s philosophy of in- 
struction; there are a number of educators who feel that there 
are many elements in conventional teaching which tend to 
prevent the best teaching. The universality of elements or 
factors cannot therefore be taken as an indication that they 
are or are not negative, or that they are or are not necessary 
conditions. Other types of investigation, including careful 
analysis, and experimentation, are important aids in furthering 
insight into causal relationships. 

Perhaps one further distinction in causal relationships 
should be mentioned. There are factors which are generally 
conducive to a certain result but which do not necessarily 
contribute to this result. For example, the amount of money 
expended per pupil by a school system does not necessarily 
affect the quality of instruction, but indirectly does so. That is, 
the system which spends more can probably have better build- 
ings, better equipment, more adequate instructional supplies, 
and better trained teachers. Again, the amount of training 
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which teachers have is itself not necessarily related to the 
quality of instruction, but may be so related. Such factors are 
to be regarded as conducive causes, rather than direct causes. 
They increase (or decrease) the probability of a certain re- 
sult, without contributing directly (perhaps we might say, 
organically) to the production of this result. They are, how- 
ever, to be regarded as falling within the category of causes. 
Many factors belong to this type and add to the difficulty 
of identifying causes, as well as increasing the general con- 
fusion concerning the nature of causes. 

One will recall from his high-school or college chemistry 
the phenomenon of catalysis. There are certain chemical re- 
actions that take place only, or more readily, in the presence 
of a catalytic agent. This agent may not enter into the reac- 
tion directly, in a chemical way, and so far as is known it is 
not necessarily changed by the chemical reaction which takes 
place; yet it is an important and sometimes an essential factor 
in bringing about a certain result. The classification of 
such causes is problematical; in some instances they may 
be regarded as simply conducive causes; in other instances 
as necessary causes, in which case they should probably be 
regarded as direct causes; and yet their structural or func- 
tional relationship to the action which takes place is varied 
and uncertain. 

Finally, it should be noticed that any given result may 
usually be produced by a number of different combinations of 
causes. For example, good teaching may consist of any num- 
ber of patterns of contributing factors. A teacher may be 
somewhat weak in certain respects but strong in others, while 
the next teacher may have other patterns of strengths and 
weaknesses, and they may, on the whole, be equally good. Or, 
various teachers may give their pupils essentially the same 
type of experiences through the use of different means. Turn- 
ing to a physiological example, we may point out that different 
persons may eat very different foods and yet be equally well 
nourished. One must ordinarily be careful to avoid the inter- 
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pretation that his discovery of certain causes precludes the 
possibility that other sets of causes might be found which 
would be just as efficient, or more so, in producing the result 
observed. In these comments, there is no intention of denying 
the possibility of discovering fundamental causes, which are 
unique and indispensable; for example, diet must contain 
certain basic elements in order to be nourishing; but causes, 
in the form of factors that are usually identified in studies, 
are not the fundamental ones which are unique and indis- 
pensable, but are rather intermediate forms that occur in 
great variety and contain in different combinations the funda- 
mental elements. Thus, many different forms of good teaching 
(as recognized by a single educational philosophy) may differ 
widely and yet may be good because they contain, in various 
and somewhat hidden forms, the elements (whatever they may 
eventually be found to be) fundamental to effective and whole- 
some learning on the part of pupils. 


THE CORRELATION METHOD 


Description of the correlation method. The correlation 
method is of modern origin,” but is already ranked high among 
the important research methods in education. It enables one to 
approach the problems of causal relationships in terms of de- 
grees of both the contributing and the dependent factors, rather 
than in terms of the dichotomies upon which one must rely 
in the use of the causal-comparative method. It enables one to 
attack problems for which only experimental procedures have 
heretofore been available, and to deal with these problems in 
terms of many variables at once rather than in terms of a 
single variable at a time. The advantages of this attack are 
many and important; its significance is still being explored. 
The correlation method means in general that with certain 
types of problems it is not necessary to depend upon strict 


22 Helen M. Walker. Studies in the History of Statistical Method, Chapter V, “Core 
relation,” pp, 92-141. Baltimore: Williams and Wilkins Co., 1929. Pp. 229. | 
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experiments, with their attendant difficulties in control of 
factors, for an analysis of causal relationships; furthermore, it 
offers certain theoretical advantages over the experimental 
technique in addition to the practical advantages, in that it 
reflects the simultaneous variation of many factors rather 
than studying the variation of a single factor when all the 
others are held constant (inoperative). This means that reality 
can be more directly approximated in one’s data. There is 
considerable ground for saying that the correlation method 
means to social science what the laboratory method does to 
physical science. 

Perhaps it should be stated here that by “correlation 
method” we do not mean simply the calculating of coefficients 
of correlation between data which may be in hand; this is 
simply a mathematical problem, which will be discussed more 
fully in the next chapter. We refer rather to an approach to 
a research problem which gathers data on many variables 
(factors) and analyzes their relationships to some result, and 
to each other, in terms of the tendency of these relationships 
to become a single-valued function. The full concept of cor- 
relation can be grasped only after a statistical study of its 
nature, secured from courses in statistics, or educational meas- 
urement, or from reading appropriate texts.** There will be no 
attempt to explain the statistical nature of correlation here, 
other than to say that all degrees of relationship may be rep- 
resented, from perfect functionality (predictability), down to 
no observable tendency for two factors to vary together. The 
correlation method of research utilizes the correlation tech- 
nique of analysis, but it also embraces the larger elements of 
a fundamental methodological approach to the solution of a 
research problem. 

23 Charles W. Odell, Statistical Method in Education, Chapter VIII, “Aa Introduction 
to Correlation,” pp. 143-50, and Chapter IX, “The Computation of the Coefficient of 
Correlation,” pp. 151-87. New York: D. Appleton-Century Co., 1935. 

Karl J, Holzinger, Statistical Methods for Students in Education, Chapter IX, “Linear 


Correlation with Quantitative Series,” pp. 141-76. Boston: Ginn and Co., 1928. Pp. 372. 
Mordecai Ezekiel, Methods of Correlation Analysis, Chapters III-V. New York: John 


Wiley and Sons, Inc., 1930. Pp. 427- 
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Measuring the strength of association. Correlation is di- 
rected primarily toward measuring the degree to which varia- 
tion in one factor, or set of factors, is associated with variation 
(in the same or in the opposite direction) in another factor. 
If this correspondence in the variation of two factors is perfect 
(in certain mathematical respects), then we have perfect cor- 
relation, indicating complete functionality, and the set of 
data representing variations in one factor can be expressed 
as a mathematical function of the data representing variations 
in the other factor. Tf, at the other extreme, there is no tend- 
ency toward systematic correspondence in the variations of 
the two factors, as measured, then there is zero degree of 
correlation, and one cannot tell anything more about one 
factor from a knowledge of the other factor than if he knew 
nothing about the latter factor. There are of course all degrees 
of relationship ranging between these extremes. Perhaps it 
should be stated that negative correlation may represent a 
high degree of functionality, but indicates that corresponding 
movements of the two factors tend to be in opposite directions 
instead of in the same direction. 

Many important investigations of association are made by 
means of simple correlation. An interesting example is to be 
found in Maller’s study *4 of the relationship between progress 
through elementary school in New York City, and various 
social and economic factors in the neighborhood from which 
the children came. He discovered that in some schools less 
than 5 per cent of the pupils were retarded, whereas in other 
schools the per cent was over so. Maller correlated this rate 
of progress for a school with various other measures of the 
neighborhood. He found, for example, that the average intelli- 
gence of the pupils had a relationship of .7o with progress. 
(The correlation coefficient ranges from zero to 1,00 or — 1.00 
as perfect positive or negative correlation; it is customarily 
written to two decimal places, but it must not be read as per 


24 J. B. Maller, “Economic and Social Correlatives of School Progress in New York 
City,” Teachers College Record, XXXIV (May, 1933), 635-70. 
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cent.) The economic status of each school neighborhood was 
secured from certain census data; this had a correlation of 
.53 with school progress. Economic dependence of families, 
as indicated by records from social agencies, bore a relationship 
of —.5o with school progress. The rate of juvenile delin- 
quency, obtained from court records, gave a correlation of 
—.43. Birth rates also showed a negative relationship, — .30 
Death rates correlated —.34. Infant mortality varied enor- 
mously from neighborhood to neighborhood; it yielded a cor- 
relation of —.46. The correlation of these six economic and 
social factors, together (by the technique known as multiple 
correlation) with school progress, was .81, which is high. 

It is interesting to note that a composite of these six factors 
representing the pupils’ environment is more closely related to 
their progress through school than is their intelligence (which 
yielded a correlation of .70). Some of the practical conclusions 
which may be drawn from such a study are of tremendous 
significance for educational administration and instruction. 
A few of Maller’s statements (footnote 24) will be quoted: 


The marked correlations between school accomplishment and factors 
of the environment indicate that the wide variability among the 
schools of the city is due chiefly if not entirely to differences in 
the nature of the pupils enrolled in the various schools rather than 
to differences in the quality of instruction in those schools... . A school 
located in a “poor” neighborhood . .. will have a slow rate of progress. 
To demand that such a school live up to an arbitrary norm is about 
as justified as demanding that all pupils be of equal height. The 
degree of progress to be expected of a given school should not be 
based upon an arbitrary standard but upon the nature of the pupils 
and the social and economic background of the neighborhood. 

When the background factors are considered, it appears that some 
schools whose records show rapid progress are nevertheless below 
the expected level, while other schools whose rate of progress is 
slow are actually exceeding the expected level of progress. To insist 
upon a standard rate of accomplishment in the latter schools is unfair 
to the pupils and the teachers. To be satisfied with such accomplish- 
ment in the former schools is unjust to the pupils as well as to the 
taxpayer. Both instances are common misapplications of democratic 
principles. 
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Further illustration of this use of correlation will not be 
given here, but the reader is referred to the “Illustrative Ma- 
terials” under The Correlation Method, in the Selected Ref- 
erences at the end of this chapter. See the studies by Gates, 
Line and Glen, Oates, St. John, and H. T. Tyler. One may also 
consult the topics, “Correlation (Statistics),” “Correlation of 
Abilities,” and “Correlation of School Subjects,” in the cur- 
rent or large cumulated volumes of the Education Index for 
many more studies based upon correlation. 

Prediction of scholastic success. One of the interesting and 
important uses of correlation is to predict the occurrence of 
certain phenomena in a certain degree. The possibility of 
doing this grows out of the establishing of relationships in 
the past which are assumed to continue to hold. If, for exam- 
ple, a certain relationship has been found to exist between 
intelligence and success in school, the prediction may be made 
that a certain pupil, having a given intelligence level, will 
have a given degree of success in school. There are, of course, 
certain cautions in connection with prediction, which every 
statistician knows, and which others should be aware of. The 
prediction is only an average prediction; it is based on the 
average degree of success of many persons in the second char- 
acteristic (as school success) all of whom had a given level 
of measurement in the first characteristic (as intelligence), 
Many of the individuals in that small portion of the total group 
of individuals studied will of course not be at the average in 
the second characteristic, and the same thing holds true for 
the individuals for whom predictions may be made—many of 
them will not run true to the average. But predicting on the 
basis of average performance, together with a statement of 
the amount of variation from this prediction that is usually 
found, is the procedure which is connected with this use of 
correlation. 

The correlation method has had extensive use in education 
for prediction. One important field of application has been 
the estimation of scholastic success. Knowing the relationship 
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between average school marks, let us say, and marks in history, 
the most probable mark of a pupil in history can be estimated 
from a knowledge of his average marks. Perhaps more fre- 
quently studies of predictability are based on the relation 
of work in a higher institution to work in a lower institution. 
Investigations of the relationship between college success and 
high-school success are regarded by colleges as important, 
because they throw light on the use of high-school marks in 
admitting students to the higher institutions. Such analyses 
have been made in large numbers. For example, Scates ?* 
studied the relationship between the high-school marks of 
pupils who entered the University of Chicago over a three-year 
period and the quality of their work in the first year at the 
University. He found a correlation of .61 for students from 
high schools in Chicago and a correlation of .55 for students 
from high schools outside of Chicago. The relationship for 
three Chicago high schools was above .7o. Further, the proba- 
bility of success in the University was calculated for students 
who might be admitted at different levels of high-school 
averages. 

Odell °° made a similar investigation from a different angle. 


25 Douglas E. Scates, “Selective Admission and Selective Retention of College Students 
at the University of Chicago,” Chapter I, “Admissions from Chicago High Schools,” 
pp. 1-75, and Chapter II, “Admissions from High Schools Out of Chicago,” pp. 76-106. 
Doctor's thesis. Chicago: University of Chicago, 1926. Pp. 383. A summary of the first 
chapter is published as “A Study of High-School and First-Year University Grades,” 
School Review, XXXII (March, 1924), 182-92. 

26 This study, in its various aspects, is reported by Charles W. Odell in four bulletins 
of the Bureau of Educational Research, University of Illinois, as follows: Conservation 
of Intelligence in Illinois High Schools, Bulletin No. 22, 1925. Pp. 55. Are College Stu- 
dents a Select Group? Bulletin No. 34, 1927. Pp. 45. Predicting the Scholastic Success of 
College Freshmen, Bulletin No. 37, 1927. Pp. 54. Predicting the Scholastic Success of 
College Students. Bulletin No. 52, 1930. Pp. 43. The third one of these four (Bulletin 
No. 37), contains the material bearing most directly and completely on the subject dis- 
cussed in the present text. The fourth one reviews the third one and gives data on in- 
terrelationships of college subjects. 

A review of literature on correlations with college success is given by David Segel, Pre- 
diction of Success in College. United States Office of Education Bulletin, 1934, No. 15. 
Washington, 1934. Pp. 98. Bibliography of 135 references. This is the most comprehen- 
sive and detailed review in this field. Summaries of findings of previous studies are given 
in Part II, pp. 57-98. E 

A briefer summary will be found in Chapter II of Odell’s third bulletin (No. 37). 
Another source of information is: Clifford Woody (Chairman), Quantitative Measure- 
ment in Institutions of Higher Learning, Chapters III and IV. Eighteenth Yearbook of 
the National Society of College Teachers of Education. Chicago: University of Chicago 
Press, 1930. 
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He studied pupils who graduated from high schools in the 
state of Illinois and who later attended colleges anywhere 
in the country. He also found a correlation of .55 between 
general high-school average and freshman average. He con- 
tinued his analysis further and found relationships between 
high-school subjects or groups of subjects and individual sub- 
jects in college, and between age and intelligence and fresh- 
man work. The correlation of freshman average with age was 
—.23, and with intelligence was .38. By combining intelligence 
with high-school average, the total (multiple) correlation with 
freshman college work was increased from .55 to .58. These 
results were based on about 2,000 students. Odell included a 
number of interesting variations in his general study, which 
are reported in the bulletins cited. He reviews the important 
investigations made up to that time (1927) and includes 
studies of the relation between elementary-school and high- 
school work. 

A similar type of attack has been made on the estimation 
of a student’s aptitude for work in a particular course or 
subject field. Such estimates have been made from general 
scholarship marks, and from marks in previous similar studies. 
Tests of various kinds have also been used. For example, one 
may make certain predictions for individual subjects from 
general intelligence measures, and he can usually do somewhat 
better from tests especially constructed to measure aptitude 
for that particular field of work. Such tests are usually known 
as “aptitude” or “prognostic” tests and are commercially 
available.2* 

For other studies in the field of predicting scholastic success, 
see the references at the end of the chapter, by Clem, 
Flemming, Gates, Krieger, Rogers, Segel, St. John, Toops, and 

27 References to bibliographies and to current sources of information on tests were 
given in footnote 11 of Chapter VII. See also, the Education Index, topics: “Prognosis 
Tests,” and “Aptitude Tests.” References to aptitude measurement in general, including 
vocational uses, were given in footnote 38 of Chapter VII. 

As a commentary on prediction one may be interested in the article by Howard Easley, 


“On the Limits of Predicting Scholastic Success,” Journal of Experimental Education, I 
(March, 1933), 272-6. 
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H. T. Tyler. See the Education Index, under topics: “Prognosis 
of Success,” with various subtopics; “Student Selection,” with 
appropriate subtopics; “Student Achievements,” with sub- 
topics; and “Aptitudes.” See, in the same source, various 
school subjects, with subhead, “Prognosis of Success” and also 
the annual U. S. Office of Education Bibliography of Research 
Studies in Education, under the topic, “Prognosis of Suc- 
cess.” 

Prediction of teaching success. Another field of prediction 
which has been explored by means of correlation is the re- 
liability with which teachers can be selected. Because of the 
importance, to the success of education, of securing teachers 
who will do effective teaching and fit in well with their en- 
vironment, investigations of this problem have been numerous. 
The correlation of subsequent teaching success with factors 
which can be ascertained in advance of appointment is one 
phase of the general problem of teacher selection—probably 
the phase that has been attacked most scientifically 

Studies of the possibility of teacher selection which are based 
on correlation are summarized in the Review of Educational 
Research, and, less completely, by Yaukey and Anderson.*° 
Barr *° reports an extensive recent study using a number of 
bases of prediction and a composite criterion of teaching 
success. Coefficients of multiple correlation (based on several 
factors combined), were secured as high as .70, which is con- 
siderably above those usually found. 

28T. L. Torgerson, “The Measurement and Prediction of Teaching Ability,” Chapter 
II, pp. 261-6, in “Teacher Personnel,” Review of Educational Research, IV (June, 1934), 
No. 3. See also p. 282 of this issue. 

R. H. Eliassen, “Teacher Selection and Placement,” Chapter IV, pp, 83-5, in ‘Teacher 
Personnel,” Review of Educational Research, I (April, 1931), No. 2. 

For a Bibliography on this subject, see Gilbert L. Betts and Others, Selected Bibliogra- 
phy on the Education of Teachers, pp. 50-2. Vol. I, National Survey of the Education 


of Teachers. Office of Education Bulletin No. 10, 1933. Washington: Office of Education, 
1932. 

29 James V. Yaukey and Paul L. Anderson, “A Review of the Literature on the Fac- 
tors Conditioning Teaching Success,” Educational Administration and Supervision, XIX 
(October, 1933), 511-20. 

30A. S. Barr, “The Measurement of Teaching Ability,” Journal of Educational Re 
search, XXVIII (April, 1935), 561-9. 

See also, William H. Lancelot and Others, The Measurement of Teaching Efficiency 
New York: The Macmillan Co., 1935. Pp. 237. 
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The selected references (Illustrative Materials) at the end 
of this chapter list studies of this problem by Coxe, Jacobs, 
Knight, Krieger, Kriner, Morris, Somers, Ullman, and Whit- 
ney. Tiegs* and Deffenbaugh and Zeigel ** give a review of 
the whole field of teacher selection. Other references will be 
found in the Education Index, under the topics, “Teaching— 
Prognosis of Success,” and “Teachers—Selection and Appoint- 
ment.” See also, for other sources, the annual United States 
Office of Education Bibliography of Research Studies in Edu- 
cation, the topics, “Prognosis of Success,” and “Teachers— 
Appointment and Tenure.” 

Prediction of population. Another field that is of great prac- 
tical importance to school administration is the estimation of 
the population—or more specifically, the estimation of the 
school population—for some years to come. Such estimates 
must be made as the basis for planning school-building pro- 
grams,** and the figures that are decided upon find expression 
in the size of buildings which are erected. Mistakes, or errors 
in judgment, are costly, whether the estimate is under or over 
the situation as it actually develops later. It is of considerable 
importance therefore that a technique be used which is as 
sound as possible. 

Engelhardt and Engelhardt ** classify the methods that have 
been used under four heads, as follows: (1) personal judg- 
ment; (2) analogy, or comparison with the rate of growth 
which other similar cities have shown; (3) multiple-factor 
method, which includes many economic and social factors 
taken as indexes of growth; and (4) mathematical methods, 
which include fitting curves by mathematical means to the 
past growth of the city and projecting these curves forward. 

31 Ernest W. Tiegs, An Evaluation of Some Techniques of Teacher Selection. Bloom- 
ington, Ill.: Public School Publishing Co., 1928. Fp. 108. 

82 W. S. Deffenbaugh and William H. Zeigel, Selection and Appointment of Teachers, 
Chapter V, “Methods of Collecting Information about Prospective Teachers,” pp. 41-74. 
National Survey of Secondary Education, Monograph No. 12, U. S. Office of Education 
Bulletin, 1932, No. 17. Washington, D. C.: 1933. Pp. 115. 

83 For references on school-building programs, see footnote 273 of Chapter VIII, 


34 N. L. Engelhardt and Fred Engelhardt, Planning School Building Programs, Chapters 
I-IV, pp. 1-90. New York: Teachers College, Columbia University, 1930. Pp. 574. 
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The third method may employ the correlation technique, as a 
means of studying its accuracy, selecting the most promising 
factors, and making forecasts. 

Engelhardt and Engelhardt offer critical comments on these 
several methods. The most crucial critical examination of them 
has been made by Chamberlain and Crawford,** who compare 
various estimates with subsequent actual developments. This 
whole field offers many opportunities for further study. Other 
references were given previously * and will not be repeated 
here. 

Correlation in fundamental research. Correlation is of very 
great importance in research concerned with fundamental 
methods and procedures. For example, in the construction of 
research tools, such as tests and other data-gathering instru- 
ments, correlation is an essential technique for checking upov 
their validity and their reliability. By validity is meant the 
extent to which the device actually does what it purports te 
do. Validity is customarily checked by correlating the results 
of the procedure with other indexes of the thing which it is 
desired to measure. Of course difficulty is generally encoun: 
tered in securing a valid criterion (the other indexes), but 
usually progress can be made in this direction, and in some 
cases the check is entirely satisfactory. Reliability means the 
degree to which the procedure can be guaranteed to give con- 
sistent results; it also is checked by correlating the results of 
an application of the device or procedure, with a second ap- 
plication. There are variations of this technique, involving use 
of the Spearman-Brown prophecy formula, which will be found 
developed in courses in statistics, and sometimes in educational 
measurements, and in textbooks on measurement. 

In recent years considerable work has been done in the 
direction of analyzing psychological characteristics (such as 
abilities) so as to reveal the fundamental constituent ele- 

35 Leo M. Chamberlain and A. B. Crawford, The Prediction of Population and School 
Enrollment in the School Survey. Bulletin of the Bureau of School Service, Vol. IV, 


No. 3, March, 1932. Lexington, Ky.: University of Kentucky. Pp. 27. 
36 See footnote 276 of Chapter VIII. 
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ments. The assumption in this work is that any response tend- 
ency of the individual can be broken up into elements, some- 
what in the manner that chemists have broken up ordinary 
substances into simple elements. According to the basic concept 
in the work, these elements are identified when measures are 
found which do not correlate with other measures. That is, 
zero correlation with other factors is taken as evidence of 
uniqueness in the factor, and it is therefore regarded as funda- 
mental and elemental. A Committee on Exploratory Study 
of Unitary Differential Traits," under the chairmanship of 
Thorndike, has been at work for some time, with a subvention 
from the American Council on Education. 

There has been a large quantity of fundamental work done 
in this field by various individuals, Spearman ** worked out 
the first basic theory, using tetrad differences, deriving a 
general factor called g, and calling his attack the “factor 
theory.” Kelley, Holzinger,*° Hotelling,** Thurstone,*? and Wil- 


87 For details see “National Deliberative Committees in Education,” p. 162, Research 
Bulletin of the National Education Association, XII (September, 1934), No. 4. 

88 Charles Spearman, The Abilities of Man. London: Macmillan and Co., Ltd., 1927. 
Pp. 4x5. (This is not his earliest work, but rather presents a matured development of 
his point of view.) 

See the series of articles by Spearman on “The Factor Theory and Its Troubles,” 
Journal of Educational Psychology, XXIV (October, 1933) to XXV (May, 1934). 

See references in the Education Index or the Psychological Index under Charles E, 
Spearman, for many other articles, 

89 Truman L. Kelley, Crossroads in the Mind of Man. Stanford University, Calif,: 
Stanford University Press, 1928, Pp. 238. 

Truman L. Kelley, Essential Traits of Mental Life. Cambridge: Harvard University 
Press, r935. Pp. 145. 

40 Karl J. Holzinger, Statistical Resumé of the Spearman Two-Factor Theory. Chicago: 
University of Chicago Press, 1930. Pp. 43. (Planographed.) 

Karl J. Holzinger and Frances Swineford, “Uniqueness of Factor Patterns,” Journal 
of Educational Psychology, XXIII (April, 1932), 247-58. 

Spearman-Holzinger Study of Unitary Traits. Preliminary reports were issued in 1935 
to the extent of five sections. Report No. 5 is “Introduction to Bi-Factor Theory.” 
Chicago: Department of Education, University of Chicago. 

For other works by Holzinger see references under his name in the Education Index. 

41 Harold Hotelling, “Analysis of a Complex of Statistical Variables into Principal 
Components,” Journal of Educational Psychology, XXIV (September-October, 1933), 
417-41, 498-520, 

2L. L. Thurstone, “Multiple Factor Analysis,” Psychological Review, XXXVIII 
(September, 1931), 406-27. 

» A Simplified Multiple Factor Method and an Outline of the Computations. 
Chicago: University of Chicago Bookstore, 1933. Pp. 25. (Planographed.) 

+ The Theory of Multiple Factors. Ann Arbor, Mich.: Edwards Brothers, 1933. 
Pp. 65. (Planographed.) 
» “The Vectors of Mind,” Psychological Review, XLI (January, 1934), 1-32. 
» Vectors of Mind, Chicago: University of Chicago Press, 1935. Pp, 206. 
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son *® have been active in this field in America, introducing va- 
rious techniques. Tryon ** and Thomson ** have probably been 
the chief critics of the various theories and methods proposed. 
The work to 1934 has been well reviewed by Flanagan,** who 
applies the Hotelling technique. A few applications are men- 
tioned in the Review of Educational Research.” Further ref- 
erences will be found in the Education Index and in the Psy- 
chological Index by consulting the authors mentioned, and in 
the former under the topics, “Statistical Methods,” “Factor 
Analysis,” and “Correlation (Statistics).” Many references of 
course appear in the works named. In general this literature is 
difficult reading and is suited to advanced students. It is cited 
here for the benefit of those who wish to devote special at- 
tention to it. A derivation of the Spearman theory has been 
given by Line and Hedman ** which should be comprehensible 
to the mathematically-minded student after a course in sta- 
tistics. 

This field probably represents the most important applica- 
tion of correlation to date, from the standpoint of pure theory. 

Cautions in the interpretation of correlation coefficients. 
The interpretation of a given degree of correlation, as indi- 

43 Edwin B. Wilson, “On Hierarchical Correlation System,” Proceedings of the National 


Academy of Science, XIV (1928), 283-91. 
, “On Overlap,” Proceedings of the National Academy of Science, XIX (1933), 


1039-44. 


, and J. Worcester, “The Resolution of Four Tests,” Proceedings of the 
National Academy of Science, XX (1934), 189-92. 

44 Robert C. Tryon, “Multiple Factor vs. Two Factors as Determiners of Ability,” 
Psychological Review, XXXIX (July, 1932), 324-51. 

Consult, the Psychological Index for other works by this writer, 

45 Godfrey H. Thomson, “A Hierarchy Without a General Factor,” British Journal 
of Psychology, VIII (September, 1916), 271-81. 
, “A Worked Out Example of the Possible Linkages of Four Correlated 
Variables on the Sampling Theory,” British Journal of Psychology, XVIII (July, 1927), 
68-76. 


, “Some Points of Mathematical Technique in the Factorial Analysis of 
Ability,” Journal of Educational Psychology, XXVII (January, 1936), 37-54. 

For other works by this writer consult the Psychological Index. 

46 John Clemans Flanagan, Factor Analysis in the Study of Personality. Stanford 
University, Calif.: Stanford University Press, 1935. Pp. 103. (Planographed.) Bibliography 
of rro references, many of which are on techniques. See especially Chapter II, “A 
Brief Survey of Available Factor Analysis Techniques,” pp. 10-27. 

47 “Psychological Tests,” Review of Educational Research, V (June, 1935), 232. 

48 W. Line and H. B. Hedman, “A Simplified Statement of the Two-Factor Theory,” 
Journal of Educational Psychology, XXIV (March, 1933), 195-220. Bibliography of 
forty-three references, on the Spearman technique. 
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cated by a correlation coefficient that has been calculated, is 
a step that properly devolves upon considerable statistical 
familiarity with the mathematics of the procedure and the 
data used. A numbet of aspects of the interpretation can, how- 
ever, be presented to call to the attention of the reader the 
more important considerations, and to indicate their general 
nature. 

First, correlation coefficients do not represent per cent. They 
are not a per cent of anything ; they are, instead, pure mathe- 
matical numbers. Second, they are probably not to be inter- 
preted in a linear sense. That is, a coefficient of .80 is probably 
not to be regarded as twice as high as a coefficient of .40. The 
scale indicating the significance of changes in the coefficient 
depends upon the particular interpretation one desires to use. 
Several mathematical functions have been derived for the pur- 
pose of interpreting the coefficient (r), among which may be 
mentioned the coefficient of alienation and the coefficient 
of determination. According to the former, y must equal .87 
before it can be regarded as a “half correlation,” or half way 
from zero to perfect correlation, and according to the second 
function y must equal .71 to be half as high as possible. If 
some other function of r were used for purposes of interpreta- 
tion, it would probably give still another value. 

Third, the ordinary correlation procedure assumes that the 
data will be linear when plotted; that is, will cluster along 
a trend line that is straight rather than curved. There are 
methods of calculating correlation for curved relationships,*® 
although they are not commonly used. If the relationship is 
curvilinear, the ordinary coefficient of correlation (known as 
the Pearson product-moment coefficient) will give a lower indi- 
cation of functionality than can be obtained by other measures. 


49 Mordecai Ezekiel, Methods of Correlation Analysis, Chapter XIV, “Determining 
the Way One Variable Changes when Two or More Other Variables Change, Using 
Curvilinear Regressions,” pp. 187-219. New York: John Wiley and Sons, Inc., 1930. 


PP ATT) Holzinger, Statistical Methods for Students in Education, Chapter X, “Non- 

Linear Correlation,” pp. 177-89. Boston: Ginn and Co., 1928. Pp. 372. 3 
Charles W. Odell, Statistical Method in Education, Chapter XIV, “The Ratio of 

Correlation,” pp. 250-9. New York: D. Appleton-Century Co., 1935. Pp. 457. 
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Fourth, errors of measurement, if variable and if not corre- 
lated with the magnitude of the measurements, will lower the 
observed (calculated) value of r. i 

Fifth, the size of the correlation coefficient depends in no 
small degree upon the range of the measurements which are 
used in its calculation. That is, a higher correlation will be 
found between the mental age of pupils and their chronological 
age if we extend the data over say eight grades of elementary- 
school work than if we use the data from a single grade, 
because the range of both mental and chronological ages is 
greater over several school grades than in a single grade. On 
the other hand, the correlation between intelligence quotients 
and marks on school work will not be appreciably raised by 
including more school grades, because neither of these charac- 
teristics is necessarily increased by extending the range of 
grades included. In fact the degree of correlation found might 
be lowered by including more grades. 

Sixth, allowance must be made for a sampling variation in 
the magnitude of r. For example, the range (variation) in the 
intelligence quotients of the second-grade pupils should, theo- 
retically, be the same as the range in grades 2 to 6 put together 
into a single group. Practically, however, variations will be 
caused by chance factors. An allowance for variation in the 
resulting coefficient which is calculated must be made because 
of chance factors that affect the original data. This is provided 
for by the measure known as probable error, a magnitude cal- 
culated to include half of the variations which are likely to 
occur because of chance factors. It will be discussed further 
in the following chapter. 

Seventh, there are various special conditions that must be 
noted. For example, if the two traits (characteristics, or 
factors), that are being correlated happen to be ratios with 
a common denominator, such as age of pupils, the resulting 
correlation coefficient will probably be spuriously high for or- 
dinary purposes, although it will not be too high as an indica- 
tion of the relationship between the ratios. That is, it must not 
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be used as an indication of the relationship between the 
numerators. 

Perhaps as an eighth and final warning, it might be said that 
series of data in biological and social sciences frequently are 
not good (completely representative) measures of what they 
are presumed to measure, so that the relationship found, 
though true within the limits of sampling fluctuation for the 
data used, may not represent at all well the relationship 
between the factors which one desires to study. 

Discussions of the interpretation of correlation are available 
in several places. Odell ° gives a lucid, readable, systematic 
treatment. Holzinger ** presents a briefer but pointed discus- 
sion. Monroe and Stuit ë give perhaps the most extended and 
systematic treatment, taking the problem up from the stand- 
point of purposes. Barr “° has pointed out certain pitfalls in 
the use of correlation. Odell ** has prepared an earlier bulletin 
on this subject. One will find other treatises in statistics text- 
books, and in articles listed in the Education Index under the 
topic, “Correlation, (Statistics).” 

Inferring causation. This section may well belong to the 
preceding discussion, for one of the cautions in the interpreta- 
tion of correlation is that it does not in itself show causation 
and cannot be so interpreted. As we have noticed previously, 
causal relationships must be established on the basis of an 
analysis which reveals a structure of relationships that will 
support the inference of causation. Once this causal connection 
has been established, the correlation coefficient may indicate 
the degree to which a factor or set of factors are effective in 
contributing to the result. 


50 Odell, op. cit., Chapter X, “The Interpretation of the Coefficient of Correlation,” 


pp. 188-208. 

51 Holzinger, op. cit., pp. 163-7. 

52 Walter S. Monroe and Dewey B, Stuit, “The Interpretation of the Coefficient of 
Correlation,” Journal of Experimental Education, I (March, 1933), 186-203. 

53 A. S. Barr, “The Coefficient of Correlation,” Journal of Educational Research, 
XXIII (January, 1931), 55-60. 

54 Charles W. Odell, The Interpretation oj the Probable Error and the Coeficient of 
Correlation. Bureau of Educational Research Bulletin, No. 32. Urbana, Ill.: University of 


Illinois, 1926. Pp. 49. 
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Barr quotes Tschuprow ** to the effect that one might cor- 
relate the size of the fire-fighting equipment sent to a city fire 
and the amount of loss from different fires and conclude (if he 
interpreted his correlation mechanically, in terms of causes) 
that, since small fire machines did less damage than large ones 
they should be sent to all fires. Freeman and Holzinger °° show 
that if one is not careful, he will be saying that such factors 
as the guality of school work and the level of the intelligence 
quotient cause a child to be a certain age. 

Holzinger points out the source of the difficulty in the 
following statement: *7 


All statistical data are affected by a multiplicity of factors which 
may obscure the meaning of the relationship found between two ob- 
served variables. For example, the correlation between high-school and 
university grades...was .612, a result doubtless due in a large 
measure to the mentality of the student. Many other factors, however, 
such as his age, sex, nationality, health, ambition, methods of study, 
regularity of attendance, and personal appearance, as well as the type 
of examinations and reaction of the instructors, doubtless contribute 
also to the observed correlation. Scholarship as measured by marks 
is thus a variable made up of a large number of other variables, and 
the correlation found is of doubtful meaning so far as causes are 
concerned, 


Odell makes the following statement: °* 


It should be emphasized that not only does a coefficient, even a high 
one, offer no proof at all that one of the two variables causes, or 
directly affects, the magnitude of the other, but also it does not justify 
the conclusion that some cause or set of causes operates directly on 
both. The only conclusion justified by the coefficient itself is that if 
it is possible to trace causal influences far enough back, enough of 
them will be found to account for the degree of relationship indicated 
by the coefficient. 


55 Barr, loc, cit. 

56 Frank N. Freeman, Mental Tests, pp. 448-50. Boston: Houghton Mifflin Co., 1926, 
Pp. 503. 

See also the more extended discussion in Karl J. Holzinger and Frank N. Freeman, 
“The Interpretation of Burt’s Regression Equation,” Journal of Educational Psychology, 
XVI (December, 1925), 577-82. 

57 Holzinger, op. cit., (footnote 49), p. 164. 

58 Odell, op. cit., (footnote 49), p. 192. 
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These two quotations warrant careful reading. They present 
a point of view concerning the successive causation of factors 
by other factors which is characteristic of the thinking of the 
mature statistician, But difficult for the beginner to grasp. 

For the purpose of isolating the effects of some of these 
secondary factors, a technique generally known as partial cor- 
relation °° is available. It has various names for special forms, 
including part correlation, semi-partial correlation, net corre- 
lation, and joint correlation. This general branch of correlation 
gives it the extreme analytical power which it possesses, That 
is, the influence of many factors upon each other can be sepa- 
rated so that the relationship of any one of them with the 
general resulting variable (commonly called in statistics the 
dependent variable) can be determined when this relationship 
is freed from the influence of any or all of the remaining fac- 
tors studied.” There are definite pitfalls in partial correlation 
to be watched, even more than in simple (called zero order) 
correlation, for all of the former elements are present and are 
augmented by the fact that the analysis is one which follows 
along preconceived mathematical lines rather than along the 
structural lines of the actual relationships, and the partialling 
out of influences may not be entirely accurate," 

The principal purpose of partial correlation is to obtain a 
more analytical measure of causation. We must always re- 
member that the fact of a structural or organic relationship 
must first be proved before we say that any factor causes or 
contributes to the causation of another; the correlation tech- 
nique in any form is appropriate only for measuring the 
strength of this relationship. One of the interpretations of this 


59 Odell, op. cit, (footnote 49), Chapter XV, “Partial Correlation,” pp. 260-82. 
Holzinget, of. cit., (footnote 49), Chapter XV, “Partial and Multiple Correlation,” 
pP. 283-316. 
4 Ezekiel, op. cit., (footnote 49), Chapters X-XVI, and XX-XXII. 

60 Jack W; Dunlap and Edward E. Cureton, “On the Analysis of Causation,” Journal 
of Educational Psychology, XXI (December, 1930), 657-80. 

61 Barbara Stoddard Burks, “On the Inadequacy of the Partial and Multiple Correlation 
Technique,” Journal of Educational Psychology, XVII (November-December, 1926), 
532-40, 625-30. 

See also, for a resumé, Odell, op. cit., (footnote 49), Pp. 278-9. 
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degree of relationship that seems to be becoming more wide- 
spread is the conversion of the correlation coefficient into a 
statement of the number of elemental factors which two vari- 
ables have in common. That is, under certain assumptions, a 
correlation of .so indicates that one-fourth of the factors 
determining the values of the two traits being correlated are 
joint factors, common to both traits. Whether the assump- 
tions are justified cannot be stated at the present time. A 
method which aspires to indicate the per cent that each of a 
number of factors contributes to a resulting variable makes use 
of path coefficients. There has been considerable discussion of 
this method in the literature, but its underlying logic is com- 
plex and its application is uncertain." Introduced by Fisher,’ 
the technique known as analysis of variance offers other means 
for causal interpretation. For those who prefer to be conserva- 
tive, however, all of these interpretations in terms of per cent 
of causation will appear to be mathematical artifacts which do 
not necessarily parallel closely the actual facts of relationship. 
More mature analysis and more penetrating research may shed 
greater light on their value. 


THE CASE-STUDY METHOD 


General characteristics and use of the case method. A 
third method for the analysis of complex causation is the case- 
study procedure. A good introductory statement of its charac- 
teristics and use follows: °° 


62 Henry E. Garrett, Statistics in Psychology and Education, pp, 291-8. New York: 
Longmans, Green and Co., 1926. Pp. 317. 

63 Odell, op. cit, (footnote 49), pp. 276-7. Gives a brief description of path co- 
efficients and references to the leading literature. 

64 R. A. Fisher, Statistical Methods for Research Workers, Chapter VII, ‘“Intraclass 
Correlations and the Analysis of Variance,” pp. 190-227, and Chapter VIII, “Further 
Applications of the Analysis of Variance,” pp. 228-62. London: Oliver and Boyd, 
Fourth edition, revised and enlarged, 1932. Pp. 307. 

George W. Snedecor, Calculation and Interpretation of Analysis of Variance and 
Covariance. Ames, Iowa: Collegiate Press, Inc., 1934. Pp. 96. 
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65 F. N. Maxfield, “The Case Study,” Educational Research Bulletin, IX (March 
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It is becoming generally recognized that in dealing in any practical 
way with human relationships and adjustments there is considerable 
advantage in developing a case-study technique. The term is well 
known in the practice of medicine and of law. The physician includes 
in the study of his “case” the history of the patient’s previous devel- 
opment, his health, and so on, and also makes a careful inventory of 
present conditions and symptoms, Recently a medical case study has 
come to include definite quantitative and qualitative data on bac- 
teriological and physiological findings. The social worker recognizes, 
too, the fundamental significance of case study and the case record 
in dealing with the complicated relationships of family situations or 
child welfare. 

This practical use of case-study technique has extended to many 
other lines of professional and business activity. The salesman is 
encouraged to make a careful case study of each prospect to whom 
he expects to sell. Data in regard to his business, credit, and trade 
policies are recorded, to be sure; but also items regarding his per- 
sonality, attitudes, and interests. Individual case studies of each mem- 
ber of a state legislature may be as important to a lobbyist for a 
large industrial corporation as the technical schedules of his own busi- 
ness. Chain stores are not placed at random but on the basis of 
careful studies of individual locations. 

The public schools, faced with the problem of mass education, 
have only gradually come to recognize the necessity for case studies 
of individual pupils. The needs of the obviously exceptional child, the 
blind, the deaf, and the seriously mentally defective, have been recog- 
nized, as well as those of the truant, Until recently, and even now in 
a majority of school districts, the other so-called “normal” children 
have been assumed to be homogeneous. Vet, recognition of the useful- 
ness of case-study technique in avoiding economic waste, in reducing 
retardation and maladjustment in school progress, and in preventing 
miscarriage in vocational preparation and guidance is gaining ground. 
The work of the medical inspector, the attendance supervisor, and the 
psychologist is being correlated and made effective by the school nurse 
and the visiting teacher who work with a case-study technique. 


Desirable characteristics of a satisfactory case study are: 
completeness of data, validity of data, continuity, confidential 
recording, and scientific synthesis (which is as much prog- 
nostic as diagnostic). A variety of blanks or forms for use in 
recording case histories may be found in the references in the 
chapter bibliography dealing with case work, especially the 
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volumes by Lundberg and Reavis. Such forms suggest the type 
of information collected in case studies. 

A case is a particular one of a kind which may be considered 
a basic unit for study.® The case may be one human being 
under investigation or an episode in his life, or it might be 
conceivably a nation, a race of people, or an epoch in history. 
The cases with which social workers are apt to be concerned 
are individuals, families, neighborhoods, and communities. The 
cases with which ethnologists, anthropologists, and historians 
work are more likely to be non-civilized tribes, culture groups, 
historical epochs, and politically organized populations. In 
educational research, cases are most commonly individual 
persons, communities, and institutions. The case-study method 
has been frequently employed in education in studying prob- 
lem cases, maladjusted pupils, and scholarship difficulties. It 
is obviously an important source of educational ideas. 

In addition to use of the case method for investigation of 
particular or individual instances of phenomena, it may be 
employed in studying the general characteristics of phenomena 
of any given class, as for example, case studies of the scholar- 
ship difficulties of secondary-school pupils, the teaching diffi- 
culties of beginning teachers, elements of likenesses and 
differences in school communities of some particular sort, etc. 
If the case method is to be used in the discovery of relation- 
ships thought to hold true for particular classes of conditions, 
then care must be taken to collect comparable data relating 
to the several cases investigated. Obviously, if one is to gen- 
eralize about the uniformities common to a number of instances 
of some phenomenon, his generalizations must be based upon 
uniform data. One of the marked limitations of many of the 
published case studies is that the data are incomplete and 
fragmentary, each report containing, in the main, merely those 
items which happened to impress the investigator most at the 

67 Howard W. Odum and Katherine Jocher, An Introduction to Social Research, pp. 
230-4. New York: Henry Holt and Co., 1929. 
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University of North Carolina Press, 1924. 


568 EDUCATIONAL RESEARCH 


time of the observation. Data collected under such conditions 
are ordinarily not very valuable for the development of valid 
generalizations about educational phenomena. 

Statistical techniques are available for analysis of data 
secured in connection with the process of group diagnosis. It is 
also possible to make statistical analyses of groups of indi- 
vidual case studies. Workers who consider case and statistical 
procedures antagonistic, or mutually exclusive, overlook the 
fact that case studies become of significance scientifically only 
when classified or summarized in some way so as to cause the 
uniformities to stand out and to group themselves into general 
patterns or types.®° 

When the case-study method is to be employed to analyze 
the common antecedents of conditions (a procedure discussed 
in connection with causal-comparative research), one must first 
make a number of individual case studies of the phenomena 
under investigation: problem cases, cases of scholarship diffi- 
culties, delinquencies, etc. Then, if the data have been collected 
in a manner to make them comparable, one may employ either 
of two methods in studying their common factors. If the 
principle of agreement (discussed in connection with causal- 
comparative research) is employed, one chooses for special 
analysis merely those instances of the phenomenon in which 
the effect under investigation is observed to be present, either 
not at all or in some marked degree. The analysis of the data, 
as has been pointed out, consists of comparing the instances in 
which the observed effect does and does not appear, for com- 
mon likenesses and differences in their antecedents. If the 
correlation method is employed, one may use cases, not merely 
at the extremes of the distribution of the effect under observa- 
tion, but all instances in which the effect may be present in 
any amount whatsoever. Coefficients of correlation may be cal- 
culated for any number of traits especially chosen for investi- 
gation. Both the principle of agreement and the correlation 


69 G. A. Lundberg, Social Research, p. 175. New York: Longmans, Green and Co., 
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technique lead to generalizations relative to the common ante- 
cedents of the phenomenon under investigation. 

The steps involved in making case studies. To secure a 
fairly accurate conception of the case-study method, some 
analysis of the problem solving or logical steps employed in 
this type of research appears necessary. The steps in thinking 
which characterize this investigational procedure are ordinarily 
given as follows:' First, either by direct observation or 
measurement, the status of the phenomenon under investiga- 
tion must be determined. This is necessary, because when one 
comes to analyze the antecedents of some observed effect, 
one studies them, not without regard for the amounts of the 
effect observed, but conjointly. If the case study is to end in 
the comprehension of a particular instance of some phe- 
nomenon singled out for special investigation, then the meas- 
urement of status is merely an aid to the identification of the 
particular condition under investigation, as for example, a 
maladjusted pupil, a “run-down” school system, or an unsuc- 

_ cessful school superintendent. Of course, one may base his 
judgment about the status of the case under study upon inci- 
dental observation, but if one expects to use the data eventu- 
ally, as is so frequently true, as a basis for some sort of a 
remedial or adjustment program, then one will ordinarily de- 
sire some more accurate measurement of the object under in- 
vestigation. If one wishes to employ the principle of agreement 
to compare the circumstance common to a number of instances 
of some particular condition, such as those already referred to, 
then of course he must determine the status of the phenomenon 
with sufficient accuracy to know’ that it is actually present or 
absent, as the case may be. If the correlation method is to be 
used in the analysis of the data relating to a number of cases 
in which the phenomenon is present in varying amounts, then 

70 The case-study method is a form of systematic diagnosis employing both the 
longitudinal and cross-sectional view of phenomena to discover how they came to be. 
The mental steps are identical with those of the better known diagnostic methods 


generally employed by field workers in the scientific study of classroom instruction: 
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the accurate measurement of the elements under investigation 
is essential. The first step then, in making a case study, is to 
determine the status of the phenomenon undergoing study. 
The second step in a case-study investigation is the collec- 
tion of data relative to the circumstances associated with the 
phenomenon under investigation. Since it is desirable that the 
collection of data arise out of something more than mere 
random observation, one will ordinarily attempt to guide this 
process by some more or less definitely formulated hypothesis 
relative to the antecedents that most probably limit the given 
condition. To secure a fairly accurate conception of the case- 
study method, one needs to understand how these hypotheses 
arise. From a study of the history of the case and a knowledge 
of similar cases, gained from diverse sources, one comes to 
some sort of a judgment about the antecedents of the particular 
effect under observation. To reach such a judgment, one ex- 
amines carefully the present behavior or status of the case and 
its historical background for symptoms of the antecedents or- 
dinarily known to be associated with such cases. Just what 
these symptoms and conditioning factors are will have been 
determined by previous experience and research with similar 
cases, symptoms, and antecedents. In making a case study one 
must identify as present in a particular instance of some con- 
dition what is known to be true of such a phenomenon in gen- 
eral. If one were investigating, for example, a problem case in 
discipline, he would know from previous experience and in- 
vestigation that problem cases in discipline may arise from 
many causes. They may result from conditions found in the 
school environment: too easy or difficult tasks, repeated failure 
in class work, special deficiencies, personality conflicts with 
the teacher, and poor school environment ; conditions found in 
the kome environment : conflicts between mother and father, the 
attitude of parents toward the child, the relations of the child 
to other children in the family, and faulty home training; and 
conditions found in the child’s play life: inability to compete 
with others on equal terms, timidity, unfortunate experiences 
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and companionship, special disabilities, etc. From a survey of 
these many probable causes of disciplinary difficulties, the in- 
vestigator must come to some judgment as to the most prob- 
able antecedents of the case under investigation. 

Thus, combined in this single step of diagnostic thinking are 
many special applications of the scientific method. In the first 
place, the approach is historical in character ; one attempts to 
understand the present status of the phenomenon by seeing 
how it came to be. In the second place, one’s judgments about 
the probable relationships involved are the products of past 
experiences and researches with similar cases, symptoms, and 
antecedents, arrived at by means of incidental or systematic 
observation, or both. In this respect the case-study method is 
the meeting point of the particular and the general, the per- 
sonal and the scientific. In the third place, all judgments about 
the antecedents of the case under investigation are hypo- 
thetical until actual measurement and adjustment have shown 
otherwise. 

The third operation in making a case study involves check- 
ing for the presence or the absence of the antecedents thought 
to apply to the situation under investigation. This step is 
necessary if the thinking about the given condition is to pro- 
ceed beyond the hypothetical stage. In the illustration given 
above, if a careful survey of the situation and its past history 
reveals that the most probable cause of the disciplinary diffi- 
culty under study is the school’s failure to adjust its program 
of instruction to individual differences, then the investigator 
will want to ascertain, as a part of the process by which the 
validation of the diagnosis is made, whether the supposed 
neglect on the part of the school really exists. In ascertaining 
this fact the worker may rely upon direct observation, objec- 
tive tests, or any other of the many means of measurement at 
his disposal. The successful completion of this step may in- 
volve the measurement of many different kinds of materials, 
products, and processes. If the condition supposed to exist is 
absent then one must return to a new study of the case; if it is 
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present, one may proceed to a further validation of the diag 
nosis by making the necessary adjustment in the instructional 
program and observing the effect. 

The fourth step in making a case study, if the diagnosis is to 
be rendered valid, involves the adjustment of the circumstances 
associated with the object of study, in some manner judged 
necessary to improve the situation. Although this fourth phase 
may be thought of as something distinct from the first three 
steps, the diagnosis cannot, however, be considered complete 
until the conditioning circumstances supposed to explain the 
case have been removed and the effect or effects observed. If 
several adjustments have been suggested by the diagnosis, and 
if one desires to know the potency of each particular factor, 
then one must modify a single circumstance at a time, with all 
others held constant, as suggested by the law of the single 
variable. 

The fifth and final step in case work, as in all educational 
diagnosis, involves the remeasurement of the phenomenon 
under investigation to ascertain what changes, if any, have 
been produced in its status by the modifications made. If the 
change is a positive one, and in an amount thought significant, 
then the diagnosis can be judged correct. 

The preceding review of the steps in making a complete case 
study emphasizes the process of educational diagnosis and 
remedial procedure in applying not only the results of case- 
study research, but also the findings of other investigational 
methods. That is, educational diagnosis is considered a way of 
utilizing or applying results secured from any or all of the 
various types of research rather than a distinct method of in- 
vestigation. The principles and practices of educational diag- 
nosis "t are thoroughly analyzed in the current literature, as 

71H. J. Baker and Virginia Traphagen. The Diagnosis and Treatment of Behavior- 
Problem Children. New York: The Macmillan Co., 1935. Pp. 393. 

L. J. Brueckner and Others, Educational Diagnosis, Thirty-fourth Yearbook of the 
National Society for the Study of Education. Bloomington, Iil.: Public School Publishing 
Co., 1933. Pp. x + 524. This year-book deals with the factors associated with learning 
difficulty (physical, intellectual, pedagogical, emotional, social, and environmental); the 


principles and techniques of educational diagnosis and treatment; diagnosis and remedial 
treatment in the various Seld: of educational responsibility and instruction (reading, Eng- 
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well as illustrated at some length in the bibliography of this 
chapter. 

Use of data-gathering procedures in case studies. There 
are three important points at which measurements are em- 
ployed in case study: (1) in determining the present status of 
the case in all of its important aspects under investigation, (2) 
in determining the presence and absence of the presumed ante- 
cedents of conditions whose status is found unsatisfactory, and 
(3) in determining the status of the various aspects of the case 
after the necessary adjustments have been made. These uses 
of measurement will involve the evaluation of many different 
kinds of products, processes, and conditions, and the employ- 
ment of many different types of procedures and instruments, 
such as tests, rating scales, check-lists, interviews, question- 
naires, etc. (as discussed in Chapters VII and VIII). In many 
instances research of this type will involve new instruments 
of measurement and extensive programs of informal objective 
measurements. If the inferences drawn from such investigations 
are to be worth while, they will, of course, need to be based 
upon accurate data. 

Insight and principles of relationship in case-study inves- 
tigation. As outlined above, the terminal point of the second 
step in case study or diagnostic thinking is a hypothesis con- 
cerning the probable causes of the unsatisfactory status of the 
phenomenon under inyestigation. The importance of having 
clear ideas in framing hypotheses has already been emphasized 
in Chapter IV, dealing with the formulation and testing of 
hypotheses. The soundness of one’s ideas depends upon the 
factors of training, experience, and capacity for this type of 
thinking. The ultimate test of one’s ideas is by making the 
adjustments in the antecedents of the phenomenon implied by 


lish, arithmetic, social studies, science, health education, behavior disorders, speech, voca- 
tional interests and aptitudes, music, art, leisure-time activities, and creativeness); and the 
administration of a program of diagnosis and remedial instruction. 

P. M. Symonds, Diagnosing Personality and Conduct. New York: Century Co., 1931. 
Pp. 602, 
, Psychological Diagnosis in Social Adjustment. New York: American Book Co.: 
1934. Pp. 362. 
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the diagnosis and by remeasuring the conditions later for evi- 
dences of positive changes in status. In this respect, the case- 
study method follows the pattern of investigation already 
identified as the steps of a complete act of thought or of prob. 
Jem solving. 

An interesting aspect of the case-study method involves the 
use of previously derived principles of relationship in the for- 
mulation of hypotheses concerning the probable causes of the 
unsatisfactory status of the phenomenon under investigation. 
The worker may use all of his own past experiences, and as 
many of the experiences of others as are available through 
direct personal contact or are recorded in history, experimental 
science, and philosophy. If phenomena similar to that under 
investigation have been subjected to previous study, the 
worker may have access to a large body of verified knowledge 
which he may employ in the formulation of hypotheses, even 
though such principles of relationship may not be fully ap- 
plicable to the particular situation under investigation by the 
case-study method. 

Publication of case materials and desirable trends. The 
situation with reference to the publication of case-study ma- 
terials might be greatly clarified, if the investigator understood 
that the case-study method may serve two purposes: (1) to 
determine the antecedents of some particular instance of a 
phenomenon, and (2) to discover the circumstances common to 
a number of instances of some condition. The first type of case- 
study should be reported to indicate that it has served the 
intended purpose. The report should contain evidence concern- 
ing the initial status of the phenomenon under investigation, a 
statement of the symptoms observed, the conclusions drawn, 
evidence concerning the supposed antecedents of the unsatis- 
factory status of the condition under investigation, the remedial 
adjustments made, and the observed effects. One might, of 
course, ask why one should ever publish individual case studies, 
Although it is true that case studies of a particular phenomenon 
were designed ñrst of all to improve some given condition, they 
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may, however, provide suggested procedures for those who deal 
with similar problems and thus, if published, further the scien- 
tific study of education. In this respect reports of the case 
study of particular phenomena are important. A report of case 
studies, made for the purpose of studying the circumstances 
common to several instances of some particular educational 
condition, should present a schedule of the antecedents studied, 
the amount of each factor present in each particular situation, 
a record of whether the antecedent was or was not judged to be 
the determining factor in each specific situation, the adjust- 
ments made, and the results secured. 

Much of the preceding discussion of the case method has 
emphasized the correction of conditions of maladjustment, but 
it should be realized that a desirable forward step would be to 
make case studies of normal and well-adjusted individuals or 
situations the basis for guidance in preventing maladjust- 
ments.” This procedure is in keeping with a trend toward pre- 
ventive medicine rather than corrective medicine. An excellent 
illustration of the way in which the case method (as one of 
several research procedures) has been applied to the study of 
bright children is the Stanford Genetic Studies of Genius, de- 
scribed in the next section of this chapter. Numerous other 
examples of the case method occur in the chapter bibliography. 


THE GENETIC METHOD 


Purposes and general characteristics of the genetic 
method, A fourth method which is used frequently in the study 
of complex causation is the genetic approach. The case method, 
as compared with the genetic approach, ordinarily involves a 
shorter segment of the life span of some subject, cultural group, 
community, or institution, studied primarily for the purpose of 


72 Elsie M. Smithies, Case Siudies of Normal Adolescent Girls. New York: D. Appleton 
and Co., 1933. Pp. x + 284. The title is not entirely accurate, since the cases reported 
involye maladjustments such as: self-distrust, physical disability, exhibitionism, volitional 
retardation, depression, insecurity, environmental pressure, parental dominance, shame, 
and inferiority fear. 
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discovering cause-and-effect relationships. The genetic method 
is applied principally to long-time investigations of biological 
phenomena where the purpose is primarily that of studying the 
change, growth, and development in these phenomena. The 
case-study method tends to be limited to the study of the 
atypical individual or situation, while the genetic method is 
more generally concerned with the study of normal life proc- 
esses. Both methods are used to analyze cause-and-effect re- 
lationships, although in some instances the genetic method 
may be devoted primarily to developmental norms. 

Although some educational geneticists disclaim any interest 
in cause-and-effect relationship and maintain that their sole 
function in the conduct of genetic studies is to record develop- 
mental changes over a considerable period of time, this is a 
limited concept of the possibilities of the genetic approach. 
Without minimizing the importance of the normative function 
of the genetic method, which has been discussed in Chapter 
VIII, the authors may insist that this procedure be employed 
in the study of cause-and-effect relationship. The situation re- 
ferred to is similar to that found in the field of history where 
many historians also insist that their sole function is that of 
preparing a faithful record of past events, and that it is the 
duty of some other group to make the appropriate applications 
of these facts to present-day problems and conditions, This 
position may be defensible from a technical point of view, but 
it is subject to two very definite limitations. In the first place, 
if such records are to be employed in a practical way, they 
must contain the information needed in solving present and 
future problems. It is only by identifying the problems of the 
present and imagining the problems of the future that the his- 
torians or geneticists may choose those facts of most worth 
from the abundance of materials at their disposal. There is 
considerable discussion in the field of genetic psychology at 
the present time, for example, about records and what type 
of information should be collected. The answer to such ques- 
tions about records will be found. if the data are to be col- 
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lected for some practical purpose either present or future, only 
by the careful study of the uses to which these facts may be 
put. A second limitation of the “faithful-record” concept of 
genetic psychology or education is that we dare not entrust 
the generalization of genetic data to those not trained in genetic 
methods, to persons who lack acquaintance with the data 
themselves. 

In the literature of education and psychology one finds ample 
evidence that many geneticists have concerned themselves, not 
merely with the “record” aspects of their field, but with im- 
portant cause-and-effect relationships. A review ** of 433 studies 
in the field of mental and physical development, dealing with 
mental growth from birth to puberty, mental and physical de- 
velopment in adolescence, physical growth from birth to 
puberty, and relationships in physical and mental development, 
includes many genetic investigations extending over a period of 
years and reveals attempts to deduce causal relationships from 
these developmental studies, for example, between: environ- 
ment and growth of intelligence, heredity and mental develop- 
ment, race and mental growth, sex and growth. dentition and 
growth, and between various anatomical, physiological, motor 
and metabolic factors. Gesell uses as a subtitle for his book, 
The Mental Growth of the Pre-School Child, the highly signifi- 
cant phrase, “A System of Developmental Diagnosis.” ™ Al- 
though the volume abounds with normative data, Gesell appears 
always to be thinking about the use of the data, and in the 
fourth and final part of his study he finally introduces an ex- 
tended discussion of developmental diagnosis. Without mini- 
mizing the preparation of accurate records of biological events, 
because these data are, after all, the foundation of all genetic 
consideration of such phenomena, the authors wish to empha- 
size the importance of the genetic method in the study of 


73G. D. Stoddard and Others, ‘Mental and Physical Development,” Review of 
Educational Research, III (April, 1933), 81-181. f 

See also G. D, Stoddard and Others, “Mental and Physical Development,” Review 
of Educational Research, VI (February, 1936), 1-152. Includes 631 references. 

74 Arnold Gesell, The Mental Growth of the Pre-School Child, New York: The Mac- 
willan Co., 1925. Pp. x + 447. 
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cause-and-effect relationships and in developmental diagnosis. 
In this latter emphasis will be found a fundamental unity which 
characterizes the various methods of research discussed in this 
chapter. 

Two types of the genetic method. In the first place, the 
genetic method may be used in the repeated observation or 
measurement of the same child or the same group of children 
over a period of years. In following the same child, the 
investigation becomes an extended case study, as it were. In 
the second instance (with a group of children), the study may 


become a series of parallel case studies, not conducted so’ 


much to understand the individual case as to discover common 
likenesses and differences among particular classes of children. 
In each of these instances the genetic method may serve a 
variety of purposes: to secure a faithful record of past bio- 
logical events in the developmental history of individuals or 
groups; to discover cause-and-effect relationships, principles of 
development, and common likenesses and differences in the 
developmental history of individuals or groups of persons; to 
determine changes in interests, capacities, and abilities from 
one age level to another for either individuals or groups of 
persons; and to secure age-grade norms of development for 
groups of children. In the attainment of these purposes the 
genetic approach may employ a variety of other methods of 
research: the historical, the normative-survey, the experimental, 
the causal-comparative, the case, and the correlation methods, 
but in so doing it maintains its position as a distinct method 
of research by reason of its purpose, pattern of organization, 
and total structure of methodology. 

In reality, a limited number of investigations have followed 
the individual child over a period of years, although such a 
procedure is of great value and is being applied increasingly 
in the laboratories and clinics for child study. A more common 
type of genetic study in the past has been to compare simul. 
taneously the stage of development reached by different age- 
groups of children. The possibility for erroneous conclusions 
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in attempting to identify genetic stages of development by 
using different age-groups of children at the same time may be 
illustrated. For example, it is known that the so-called adoles- 
cent spurt of growth in height does not commence in all chil- 
dren at the same age." 


In girls the usual time falls between the ages of eleven and four- 
teen, Some girls aged thirteen years will be in the midst of the rapid 
growth period, while others will not have entered it, and still others 
will have passed beyond it. To take the average increment and say 
that it is typical for thirteen-year-old girls is misleading. A pre- 
adolescent girl should be expected to have a much smaller increment 
of growth than one who is passing through the period of most rapid 
growth. Mental growth is no doubt characterized by similar, but sub- 
tler complications. 


Though it is true that with the more common cross-sectional 
method of selecting a sampling, trends between groups may be 
studied, the longitudinal approach, “whereby children are 
brought in for repeated measurements or experiments over a 
long period of time, holds certain values which are lost in the 
cross-sectional scheme. The longitudinal or genetic approach 
simplifies the statistical problems immensely and may result 
in certain unique discoveries.” 

In addition to standardized tests of various types, some in- 
genious techniques of measurement and recording have been 
developed for the use of investigators who are studying growth, 
especially in the field of physical development: stereoscopic 
photographs, measurement of the mass of the body and its 
segments by water displacement, X-rays, etc.” 

Examples of genetic studies. Among the studies dealing 
with mental and physical development summarized in the Re- 
view of Educational Research, specific mention may be made 
of the works of Terman, Gesell, and Baldwin, which provide 

75 G. D. Stoddard and Others, op. cit., 


76G. D. Stoddard, “Methods of ase Ja Child Psychology,” in “Methods and 
Technics of Educational Research,” p. 70, Review of Educational Research, IV 


(February, 1934), 1-119. k è 
77G. D. Stoddard and Others, “Mental and Physical Development,” Review of 


Educational Research, TII (April. 1933), 81-181. 
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good illustrations of the genetic type of investigation.’* In cer- 
tain respects these are also case studies. 

This type of research may be made more realistic by review- 
ing briefly the Stanford Genetic Studies of Genius. This series 
of investigations illustrates effectively the way in which 
a number of research methods may be combined to accomplish 
a given purpose. In these genetic studies, data were secured 
through historical, normative-survey, causal-comparative, cor- 
relation, and case procedures, as parts of the total genetic 
approach, and applied to developmental and growth problems. 


First ILLUSTRATION "° 


The purpose of the study was to determine the traits which charac- 
terize children of marked incellectual ability. The gifted children were 
selected on the basis of nomination and tests, the work being carried 
out with great care and thoroughness by a staff of trained assistants. 
As the expense involved made it necessary to confine the selection to 
the largest cities, the gifted children were distinctly of the urban 
type. A staff of trained workers administered physical and mental 
tests, made medical examinations, and secured character and person- 
ality ratings for gifted children and a control group, The net result 
was that the gifted children equaled or excelled the normal control 
group in physical growth and general health as well as in the various 
scholastic and social ratings. In brief, the gifted child appears to be 
an all-round individual, contrary to the common opinion that intellec- 
tual precocity goes with physical defect and social instability. 


SECOND ILLUSTRATION 8° 


This book describes the methods, and presents the findings, of an 
attempt to estimate the I.Q.’s of three hundred geniuses born between 


78B. T. Baldwin, The Physical Growth of Children from Birth to Maturity, University 
of Towa Studies in Child Welfare, Vol. I, No. x. Iowa City, Iowa: University of Towa 
1921. Pp. 412. i 
Arnold Gesell and Others, An Atlas of Human Behavior, Vols. I and II, New Haven. 
Conn.: Yale University Press, 1934. Pp. 922. 4 
g aor Gesell, Infancy and Human Growth, New York: The Macmillan Co., 1928. 
P. 418. 
L. M. Terman and Others, Genetic Studies of Genius, Vols. I, II, III. Stanford 
Calif.: Stanford University Press, 1925, 1926, 1930. Pp. 648, 842, $08. 5 
79 Lewis M. Terman and Others, Genetic Studies of Genius, Vol. 1. Stanford University, 
Calif.: Stanford University Press, 1925. Pp. xvi +648. The present summary statement 
is adapted from a review by K. J. Holzinger, Elementary School Journal, XXVI (Janu. 
ary, 1926), 387-90. 
80 Catharine Morris Cox, Genetic Studies of Genius, Vol. II. Stanford University, 
Calif.: Stanford University Press, 1926. Pp. xxiv + 842. The present summary statement 
is adapted from a review by C. S. Hobson, School Review, XXXV (April, 1927), 516.8, 
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1450 and 1849. The biographies of the geniuses selected for the study 
were searched for early indications of superior mental ability and for 
significant environmental influences. Two periods in the life of each 
subject were studied: (1) childhood and youth to the age of seventeen 
and (2) early manhood or womanhood from the age of seventeen to 
the age of twenty-six. The social standing of other members of the 
family and the early activities of the subject himself were considered 
significant in rating the genius. His performance as represented by the 
data in his biographies was compared by each of four judges with 
the age norms of the Stanford-Binet test. From such a comparison, 
the I.Q. that would most reasonably account for the early activities 
disclosed by the biographies was determined for each of the two 
periods in the life of the genius. The editor explains that the 1.Q.’s 
thus found belong to the biographical data rather than to the subject. 

To secure an estimate of the I.Q. of the subject, the author con- 
sidered the reliability of the data, the reliability of the age norms 
of the Stanford-Binet test, and the reliability of the individual judges. 
Reliability was based on only partly objective standards, Coefficients 
of reliability were computed mathematically. The two 1.Q.’s ascribed 
to the data on any particular genius were then corrected by the use 
of the coefficients of reliability, and a “Corrected 1.Q. Estimate” was 
made for him. 

A sampling of one hundred subjects was taken for a special study 
of the significant environmental influences on the character traits 
of geniuses. Each subject was rated on sixty-seven character traits. 
Each trait was made explicit by a brief description, A reliability 
coefficient of the ratings on the character traits was calculated for 
each subject. A comparison of ratings with 1.Q.’s was made to deter- 
mine the significance of traits of character other than brightness in 
accounting for genius. 

The first 219 pages of the book are devoted to an analysis of the 
preblem, a description and justification of the methods used, descrip- 
tions of the subjects chosen, explanations of the statistical procedures 
involved, and a presentation in full of the findings. The remainder of 
the book is devoted to brief case studies of all the subjects, “A Case 
Study in Full,” “Excerpts from the Early Writings of Young 
Geniuses,” and the indexes. 


THIRD ILLUSTRATION 8t 


The volume is divided into three parts. Part I reports the results 
of a second survey of the thousand children described in Volume I; 


s1 Barbara Stoddard Burks, Dortha Williams Jensen, and Lewis M. Terman, Genetic 
Studies of Genius, Vol. IIL (The Promise of Youth: Follow-Up Studies of a Thousand 
udia Children), Stanford University, Calif.: Stanford University Press, 1930. 
Pp. xiv + 508. The present summary statement is adapted from a review by Margaret 
Cobb, Elementary School Journal, XXXI (April, 1931), 628-9. 
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Part II consists of case studies, accounts of individual children which 
illustrate their possibilities under different conditions; Part III deals 
with the literary ability of some members of the group who from an 
early age have been interested in writing. 

The first two chapters in Part I explain the tests and procedures 
used. The chapters following deal with intelligence; educational 
progress and achievement; school marks and honors; scholastic and 
vocational interests; social, personality, health and home tests and 
ratings; family statistics; and follow-up reports of the original high- 
school and special-ability groups. 

Part II consists of a series of fascinating case studies, illustrating 
cases of brilliant school achievement; drop in intelligence and in school 
achievement; reactions to various degrees of acceleration in school; 
effect of high intelligence on social relations of children; and the 
conquering of, or failure to conquer, environmental difficulties by 
children of superior endowment, There are also included case studies 
of twins and of siblings, both like and unlike in their high intelligence; 
of children who presented problems of behavior and personality, in- 
cluding one youthful suicide; of children of great musical ability; of 
young collectors and scientists; a babyhood study; and other high 
lights. 

Part III is an interesting application of the methods of the study 
reported in Volume II (The Early Mental Traits of Three Hundred 
Geniuses) to a study of the best literary productions of these Cali- 
fornia children. In this connection it was found necessary to make a 
rating scale for judging superior juvenile writings. By use of this 
scale the youthful writings of various great writers from Milton to 
Holmes were rated along with the best of those of the California 
young people. Seven of the California children (all girls) and twenty- 
five of twenty-eight great writers had, as children, produced highly 
superior work. The scale itself is Presented with sample passages and 
the ratings given them, and it is followed by case studies of the seven 
girls and of the ten great writers whose early lives had not been 
included in Volume II. No other characteristic of these young writers 
could be said to predict ability to write, actual accomplishment in writ- 
ing being the only reliable predictive measure. 

It is suggested that another survey be made ten years hence, Much 
might be learned from a survey once every ten years during the life- 
times of these California children whose “promise of youth” is now 
so thoroughly on record. We need to know what portion of it will be 
fulfilled and how much more nearly it might have been fulfilled under 
wiser treatment in home and school, 


Concluding statement. It has been the purpose of this chap- 
ter to discuss a number of the methods of educational research 
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which are particularly adapted to the analysis of complex causal 
relationships. These are the causal-comparative, correlation, 
case-study, and genetic methods. They present new and impor- 
tant procedures for studying educational problems. These 
methods are particularly useful in the investigation of phenom- 
ena under uncontrolled conditions and hence are peculiarly 
fitted for the study of problems in biology, psychology, eco- 
nomics, sociology, and education. The causal-comparative 
method starts with observed effects and seeks to determine the 
causal factors involved. It compares different situations by 
noting the factors which are present in those instances that 
produce a given result, and which are absent in other situations 
that fail to produce this result. The correlation method makes 
it possible to approach the problems of complex causal rela- 
tionships in terms of degrees of both the contributing factors 
and the resulting or dependent factors. It enables the investi- 
gator to deal with a number of variables at the same time 
rather than to be limited by the law of the single variable as in 
experimental work. Although the case-study procedure was 
developed to investigate individual cases, it may be used in 
the analysis of general phenomena or conditions pertaining to 
a given class or group. The case-study method, when syste- 
matically applied to the analysis of general phenomena, has 
certain advantages over the other research procedures and may 
make important contributions to the science of education. The 
genetic method is important in that it presents longitudinal or 
developmental views of biological aspects of education. It is 
significant that a recent text in psychology ® is written from 
the genetic point of view, covering the behavior of human 
beings from the prenatal period to old age, and that an 
important body of research studies ® dealing with the develop- 
mental aspects of even the prenatal period has accumulated. 

82 Florence L, Goodenough, Developmental Psychology: An Introduction to the Study 
of Human Behavior. New York: D. Appleton-Century Co., 1934. Pp. xvii + 619. 

83 Carl Murchison, ed., A Handbook of Child Psychology. (Second edition, revised, 
1933.) Worcester, Mass.; Clark University Press, 1931. Pp. 712. Pp. xii + 956. Extensive 


bibliographies and summaries of the literature in this field. A discussion of the genetic 
method in psychology with special reference to prenatal life may be found on pp. 31-5. 
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PROBLEMS AND EXERCISES 


1, Select one example of each type of research described in this 
chapter, with a date of publication not more than.two years old at 
the time this exercise is undertaken, and describe the problems and 
procedures briefly. 

2. Choose one of the studies used in the preceding exercise and 
evaluate it in terms of the principles developed in this chapter. 

3. Select some important group of genetic studies, such as those di- 
rected by Terman at Stanford or by Gesell at Yale, and identify 
the different research procedures employed as phases of the genetic 
approach. 

4. Select a problem which might well be attacked by the causal-com- 
parative method, and outline briefly an appropriate research pro- 
cedure for dealing with it. 

5. Examine three simple (zero order) correlations in different studies 
(such as in the illustrative material at the end of the chapter), and 
attempt to analyze the two traits that are correlated in each case, 
in order: (1) to determine what factors may lie back of the traits 
used so as to be responsible for (increasing, or lowering) the cor- 
relation between the traits, and (2) to determine whether there is 
any justification for regarding one trait as in part the cause of 
the other, and why. 

6. The various research methods recognized in Pih text have now all 
been presented. Examine six theses (presumably those on file in 
your institution) to see what method or methods have been used 
in each case, and classify the various studies on the basis of the 
predominant method, describing the study enough to justify your 
classification. 
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CHAPTER XI 


ANALYSIS AND INTERPRETATION OF DATA 


Introduction. The various methods of attacking problems 
have been presented in the immediately preceding chapters. 
The present chapter treats the next steps in research, namely, 
the analysis of the data after they have been collected, and the 
interpretation of these analyses. Since these steps are largely 
dependent upon the particular method which has been pursued 
and upon the data that have been gathered, a complete treat- 
ment cannot be given here. Discussions have already been 
presented in preceding chapters of certain problems of analysis 
and interpretation peculiar to the methods treated in such 
chapters. The purpose of this chapter will be to call attention 
to the relatively general aspects of analysis and interpretation 
which are common to most quantitative studies. 


PROCEDURES IN ANALYSIS 


Analysis required from beginning of work. Although this 
chapter is generally concerned with the analysis of data after 
they have been collected, analysis is a process which enters into 
„research in one form or another from the very beginning. When 
one selects a problem to work on, and begins planning his at- 
tack upon it, he at once engages upon an analysis of the total 
problem in order to formulate a hypothesis which will enable 
him to start work with some plan, and in order to begin gather- 
ing data that will fit in with his hypothesis. That is, determin- 
ing the definition of the concepts which are involved in one’s 
study calls for an analysis of the field conditions and of the 
factors likely to be reflected in the data if various definitions 
are adopted. One will further analyze the field and his problem 
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in determining the factors for study in the general problem, 
and he will engage in more analysis when methods of measur- 
ing these factors are being considered. It may be fair to say 
that research consists in general of two large steps—the gath- 
ering of data, and the analysis of these data; but no amount 
of analysis can (validly) extract from the data factors which 
are not present. Consequently, the final analysis must be antici- 
pated in considerable detail when plans are being made for 
gathering data which will reflect the requisite factors, and this 
calls for analysis of several kinds at the start ot one’s work. 

The analysis required in these early stages may need to be 
just as penetrating as that which follows the gathering of the 
data. Indeed, one might say that it is in reality more important, 
for the later analysis assumes that the data in hand do ac- 
tually represent certain things. It is the responsibility of the 
worker to see that his early analysis of his problem, and of its 
requirements as it is subsequently developed, is sufficiently 
valid so that the later interpretation is not built mechanically 
upon an empty set of facts. 

It might be said that this early analysis is concerned with 
four things. In the first place, one must analyze his problem 
carefully to see what is necessary to provide a solution of it. 
The worker must assure himself, and be able to satisfy those 
to whom he reports his study, that his method of attacking the 
problem provides a crucial approach.* That is, he must be cer- 
tain that the method will really get at the problem and afford 
a definite answer in terms of yes or no, or in terms of definite 
degrees. In the second place, one must see that the factors 
which he chooses for study will satisfy the conditions of his 
problem—a more detailed step than the first one. He should 
ask himself: “Am I studying enough factors or elements te 
yield a satisfactory answer to my problem?” “Have I selected 
elements for study which seem reasonably promising under 

1-McCall discusses this problem in relation to two research methods, the causal 
comparative and the experimental. See William A. McCall, How to Experiment in 


Education, Chapter X, “Analyses of Experimental and Causal Investigations,” pp. 
245-69. New York: The Macmillan Co., 1923. Pp. 281. 
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critical examination?” “Have I by this selection excluded cer- 
tain factors that might be of genuine significance in this 
problem?” 

In the third place, one must examine his source of data 
carefully to see that the factors in which he is interested will 
have an opportunity to demonstrate themselves. Are the limits 
which he has set for his source-field (time period, area, or other 
limiting conditions) sufficiently hroad to permit ample oper- 
ation of the factors to be studied? If not, one may get only the 
minor deviations from the trend of large factors, which will 
mislead him. On the other hand, are the limits of the source- 
field too broad? Will many spurious factors enter and perhaps 
cause sufficiently different conditions within the source-field 
so that the important factors will “average out,” or neutralize 
each other? In the fourth place, one must examine the means 
which he expects to employ in gathering data, to see that these 
means are capable of registering variations of appropriate mag- 
nitude, simplicity (purity, or singleness), and at the same time 
complexity. 

As an illustration of the application of these considerations, 
we may assume that one wishes to study the causes of failure 
in school work. His early analysis will be concerned with such 
questions as the following: “What kind of an attack will throw 
light on the answer to this problem?” “Should I employ the 
historical method, the normative-survey, the experimental, or 
some one of the other methods?” The answer will of course 
depend in part upon the type of study which is to be made; 
different aspects of a problem may be attacked by a variety 
of different methods. One cannot hope to make a complete 
study of any problem, so that he exhausts all possibilities of 
studying further aspects of it. Turning to the factors to be 
studied, one may think of absence, intelligence, previous suc- 
cess or failure in school work, etc. Again the answer will de- 
pend in part upon what kind of study one plans to make—its 
degree of penetration or superficiality. If one is content to say 
that absence is a factor in school success or failure, he is mak- 
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ing a relatively superficial study. If he attempts to measure 
a variety of factors entering into the lack of adjustment be- 
tween the school work, as actually carried on in the classroom, 
and the habits, interests, abilities, outlooks, and personal sensi- 
tivities of the pupils who are failing, he is making a more 
penetrating—and more significant—study. There have been far 
too many studies of the first sort, correlating failure with 
amount of absence, when what education needs is an analysis 
of the factors which lie behind absence and tend to make 
school work a succession of daily defeats for the pupil. 

In the determination of the limits of the field from which one 
will gather his data, various practical factors will enter, as well 
as theoretical ones. If the study is to be normative-survey, one 
will gather data over a wide area; if it is to be historical, one 
will secure data over a period of years. The procedure is defi- 
nitely affected by the type of study contemplated. In any case, 
one will have to consider carefully what is likely to be the 
result if he includes schools of many different kinds, a wide 
variety of school grades, pupils of different levels of intelli- 
gence, etc. Finally, when consideration is given to the gathering 
of the data, one must examine his instruments. What is to be 
considered school failure? Will the evidence used exclude the 
factor of the social and political standing of the parents in the 
community? If tests are used, do they measure what the par- 
ticular school is trying to do? Do they measure what the 
teacher has covered? Do they measure effects of what the 
school is doing in an unusual way to develop the pupil in all 
aspects of his growth, or are they concerned only with academic 
knowledge demonstrated under formal classroom conditions? * 
One cannot get satisfactory evidence of factors which are not 
adequately provided for in the instruments which he uses.® 


2See the discussion of measuring many aspects of personal development, given in 
Chapter VII in connection with survey testing. See footnotes 28 and 45-58, of Chapter 
VII. Further treatment occurs in connection with observation and appraisal, in Chapter 
VOL 
3 Douglas E. Scates, “Complexity of Test Items as a Factor in the Validity of 
Measurement,” accepted for publication in the Journal of Educational Research, 1936. 

Ralph W. Tyler, “Assumptions Involved in Achievement-Test Construction,” Educa- 
tional Research Bulletin, YTI (February 8, 1933), 29-36. Also in Constructing Achieve- 
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One should not get the impression that all of these decisions 
can be made in advance. It is necessary, from the standpoint 
of systematic and effective work, that one formulate a plan 
which will direct his main efforts; but one must always be on 
the alert for cues which would indicate that modifications of 
the plan are desirable, or even that the plan does not promise 
any returns of worth: The shifting of attention and the follow- 
ing up of leads have been well described for one problem by 
Brownell.t Beginners in research would be greatly aided by 
more such detailed, frank accounts, portraying the difficulties, 
the uncertainties, the perplexities of the research worker, as he 
engages in the analysis of a difficult problem. The general pic- 
ture is given by Brownell at the outset of his description: ° 


The report is divided into two parts. . . . The first part is an ac- 
count of the steps in the investigation. It is... concerned with the 
way in which the problem was almost accidentally discovered and with 
the manner in which through the application of appropriate research 
techniques the problem, none too clear at times, was finally made to 
yield to a solution. This part of the report . . . is to furnish a realis- 
tic example of the way in which much research is done: with periods 
of alternate confusion and enlightenment, with the commission of 
many errors of judgment and the correction of those errors, and with 
the following of faulty leads and the attainment of fortunate suc- 
cesses. (Such a description of research method, it will be noted, bears 
little resemblance to the descriptions to be found in manuals and in 
many theoretical discussions of research.) The second end served by 
this part of the report is to supply evidence that not all research is 
spiritless grubbing and boring drudgery, but that on occasion it affords 
all the excitement and all the thrills enjoyed by a detective at work on 
his favorite type of crime. 


The discussion which has been presented thus far in the 
chapter is related to analysis in advance of that which is done 
after the data are all in hand. This discussion has been pre- 
sented because analysis during the early steps in research is a 
necessary foundation for the analysis which is to follow. Many 


ment Tests, a collection of planographed reprints by Tyler. Columbus: Bureau of 
Educational Research, Ohio State University, 1934. Pp. 102. 

4 William A. Brownell, “An Evaluation of an Arithmetic ‘Crutch, ” Journal of Es- 
perimental Education, II (September, 1933), 5-34. 

5 Ibid., p. 5. 
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a graduate student has found that he had to change his prob- 
lem after he had his data collected and began analyzing them, 
because he had not collected data in answer to the principal 
questions in his original problem. By appropriate thought in 
advance, one can not only avoid such enforced changes, but he 
can avoid the forcing of interpretations on his data which they 
do not legitimately support. 

Getting started on the analysis of data. It is a rather com- 
mon experience for those engaged in research to gather a great 
many data, to have the data carefully arranged on sheets of 
paper, and then to find themselves at a complete loss as to 
what to do next. A few suggestions will be given which may aid 
one in passing through this stage of his work somewhat more 
easily. It might be said, however, that such a stage of con- 
fusion, while very unpleasant and baffling, is not to be regarded 
as evidence of the inability of the worker ; it is, in fact, some- 
what typical of research, and even the most experienced in- 
vestigator will find many points in his work where he is at a 
loss to determine what the next step is, if he is undertaking a 
difficult problem. 

One mode of attack which sometimes proves helpful is to 
think in terms of tables. One may look at his data and ask, 
“What tables of significance could I make up out of these 
data?” © “In what systematic arrangements of these data can I 

6 Treatises on the preparation of formal tables may be suggestive to the student in 
starting this type of analysis. See the following: 

Bruce D. Mudgett, Statistical Tables and Graphs, pp. 3-58. Boston: Houghton Mifflin 
Co., 1930. Pp. 194. 

Helen M. Walker and Walter N. Durost, Statistical Tables: Their Structure and Use. 
New York: Teachers College, Columbia University, 1935. Pp. 96. 

Walter S. Monroe and Max D. Engelhart, The Techniques of Educational Research, 
pp. 46-48. Bureau of Educational Research Bulletin, No. 38. Urbana, Il.: University of 
Mlinois, January 10, 1928. Pp. 84. 

Most books on statistical methods have a chapter dealing with tabulation and classifi- 
cation of data. For example, see Charles W. Odell, Statistical Method in Education, Chap- 


ter II, “Tabulation and Classification,” pp. 14-31. New York: D. Appleton-Century 
Co., 1935. Pp. 457- DAA ; 
' See also Chapter XIII of the present text for a discussion of tabular presentation of 
data, with emphasis on the form of tables. 

Two references on the preparation of typewritten tables may be given here: 

Bernhard Bargen, Tabulation Technique. New York: Gregg Publishing Co., 1934. 
Pp. 46. 

EH. Brown, Statistical Typewriting. Monograph Series, No. 14. New York: The 
Ronald Press Co., 1924. Pp. 77+ 
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present a picture that is interesting in connection with my 
general problem?” This type of thinking is approaching the 
matter from the mechanical aspect, rather than from the stand- 
point of purposeful planning, but it will often help one to get 
started, and after the “ice is broken,” one may see other com- 
parisons, or points which he feels should be brought into relief. 

A careful examination of the statement of the problem, an 
inspection of the earlier analyses, a study of the original records 
of data, may afford some suggestion of interesting (significant) 
aspects that one should immediately work toward in the re- 
arranging and possibly in the statistical analysis of his data. 
Merely writing up a few case histories of individual cases (in 
almost any kind of study) may stimulate one’s mind to think- 
ing about comparisons and other relationships which should 
be studied in his data at large; if certain facts are obvious in a 
particular case, are they so in general? This attack is similar 
to the process of analyzing more or less complex arithmetic 
problems by thinking of them in terms of small numbers, where 
the answers and the relationships are rather obvious. 

Still another approach is for one to get away from his data 
and to think about the problem as well as possible from the 
point of view of one who has not been working on it. What 
are the aspects that such a person would want to know about? 
What questions would such a person raise? And what further 
questions are there about the answers to these first questions? 
Usually it is very helpful to discuss the problem in this way 
with some one who has a stimulating interest in the work, but 
who is not buried in the details of it. 

It may also be helpful to attack one’s data by making various 
simple statistical calculations. Should some data be expressed 
as per cents of others? Are the data too finely classified—e., 
are they subdivided too many times on the basis of a number 
of factors at once? Sometimes the elimination of several factors 
so that one can study the distribution of cases on a single factor 
is an aid in securing a better picture. Should the data be 
grouped according to an individual, such as several test scores 
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for each pupil, or grouped according to some other basis, as 
grade, subject, school, etc? A number of statistical procedures 
are presented in the next section. Simply applying these 
mechanically will often reveal interesting aspects that are 
worth while. 

One will of course realize that these suggestions of mechani- 
cal modes of attack are merely for beginning the analysis, and 
that before one finishes his analysis and presents the results 
and conclusions, he must have a justifiable reason for what is 
done. That is, he will probably discard much of his early 
analysis, keeping only those portions for whose retention there 
is an important reason. After having made a start, one’s ana- 
lytical procedures will be guided by his findings through pre- 
ceding analyses, as well as by the principal problem he set out 
to solve. It may be, also, that his exploration will discover a 
problem of equal or even greater importance than the one he 
set out with, and a change in the original statement may be 
justified. 


STATISTICAL METHODS OF ANALYSIS 


The field of statistical methods has been developed phe- 
nomenally within the last half century in response to a wide- 
spread need for analytical procedures appropriate to dealing 
with large numbers of cases. This section of the present chap- 
ter will call attention to the more common modes of statistical 
analysis and indicate how they fit into the general process 
of analysis. 

Throughout this text, when dealing with highly specialized 
techniques of procedure, such as test construction, interview- 
ing, the correlation method, etc., the student has been referred 
for technical details to texts which are devoted entirely to the 
subject. The same practice is followed here. It is not the 
thought of the authors that this treatment should provide 
training in statistics. In fact there are strong reasons for pre~ 
ferring not to go into such a specialized field, in addition to 
limitations of space and instructional time. Chief among such 


602 EDUCATIONAL RESEARCH 


reasons is the desire not to distort the emphasis in the total 
research picture. Students (and often instructors) become so 
concerned with the details of statistical procedures in the ab- 
stract that they lose sight of the larger background of thought 
and purpose into which these procedures must fit. It is the plan 
of the present book, being concerned with research in the large, 
to deal with the development of a general background for re- 
search. A concentrated emphasis upon the narrower, technical 
aspects of certain portions of research, such as statistical 
methods, is the province of specialized texts. The present treat- 
ment is concerned merely with giving the general concepts of 
statistical tools and with indicating roughly their uses. 

Ranks and percentile ranks. A rank is merely the relative 
position which any particular case occupies in a group. Some- 
times groups are ranked from the top (highest value) down- 
ward, and sometimes from the bottom (lowest value) upward, 
the first measure in either direction being called the first 
rank, the second measure, the second rank, etc. This type of 
information is often of very real interest; perhaps it is the 
most common numerical aspect of facts outside of their 
enumeration and measurement. Often it is of more significance 
than the measurement itself, for it contains a strong compara- 
tive element. For example, we are (or used to be), more con- 
cerned with the fact that a certain pupil is at the head of his 
class, with the fact that a certain college is at the head of the 
football league, etc., than we are in knowing the particular 
score or the number of games won. 

The significance of a rank is, of course, in part dependent 
upon the size of the group; a rank of ten is near one end of 
the distribution in a group of 1000, and it is near the other end 
of the distribution in a group of 11. To overcome this difficulty, 
the idea of percentile rank was developed. Percentile rank is 
essentially the rank that one would have in a group of roo. It 
is always calculated from the bottom upward, so that a per- 
centile rank of z means (approximately) the lowest in the 
group (of 100), a percentile rank of 40 means that 40 per cent 
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of the group lie below (the midpoint value of) this case, etc. 
Percentile rank is merely a device for standardizing the num- 
ber of cases on which the rank is calculated, so that it will 
have a stable significance. 

Measures of central tendency. There are several forms of 
averages, the most common being the mean. Even in statistical 
usage, however, the word “average” may be used in a narrow 
sense to indicate only the mean and not the other kinds of 
average. The median and mode are the other common forms of 
average. The mean is secured by adding up the values in a 
series and dividing the total by the number of cases; this is 
commonly understood by most people, since it is usually taught 
in the elementary school. The median is the point (or value) 
that divides the cases in half; that is, into two equal groups. 
As many cases lie above the median as below it. (Statistics 
has special methods for dealing with cases that fall at the 
median.) The significance of such a point will be apparent at 
once, The mode is simply the value where the greatest number 
of cases fall; usually it is not a very stable value, since the 
movement of a single case might disturb it considerably. 

The significance of these three different kinds of average 
may be clarified by relating them to a frequency distribution. 
(A frequency distribution is simply a series of values that have 
been tabulated, or tallied, opposite a scale made up of units, 
or groups of units, for the purpose.) If such a distribution, 
made up of tally marks, were cut out of a piece of paper or 
cardboard, and a pin were put in the edge along the scale at 
a point where the distribution would just balance, that point 
would be the mean. If the pin is moved to the point where 
the median falls, the distribution will not balance (unless it is 
symmetrical, and the mean and median fall at the same point), 
but a line erected at that point perpendicular to the scale will 
divide the area in half. If the pin is placed along the scale 
where the mode is, neither of the above things will be true 
(unless the mode coincides with one or both of the other 


averages). 
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The mode does not receive much use in educational prob- 
lems. The median is popular, in part because of its significance, 
and in part because it is readily calculated. It is not as sensi- 
tive to the influence of measures at the extremes as the mean 
is. The mean is mathematically the most significant average 
and is used most extensively in advanced statistical analysis. 
The investigator should, however, be careful to avoid the idea 
that there is any one best average; each one is best for cer- 
tain particular purposes, and the worker selects his measure 
to suit the purpose. Frequently there is not a great deal of 
difference between them; when the distribution is symmetrical, 
all three measures fall at the same point. 

One must of course remember when using an average to rep- 
resent a distribution, or series, that, after all, this is only a 
central measure, and although it may represent the entire 
group as well as any single value can, the series is nevertheless 
spread out on either side of this average. People periodically 
“discover” that averages do not have all the significance of the 
full distribution, and one will frequently hear people say that 
averages are misleading. That of course depends on how one 
thinks of averages; they are misleading if one conceives of the 
whole series as being fully epitomized in one figure. 

Measures of dispersion. Dispersion indicates the spread of 
the series. This affords the most important corrective for too 
liberal a concept of the average. If one knows the extent of 
spread in the series, as well as the mean, he is not so likely to 
overlook the fact that the mean is merely a middle value and 
is limited in its powers of representation. 

One can collect all sorts of curious and amusing stories 
illustrating the foolishness of depending too largely or inap- 
propriately upon the average. For example, the hypothetical 
story is told of the doctor who took the temperature of ten of 
his patients, averaged the readings, and said that all of them 
were all right because the average of their temperatures was 
normal. A measure of dispersion would prevent such a conclu- 
sion. Or, one would scarcely put a springboard over a swim- 
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ming pool that was strong enough only to support a pupil of 
average weight. A measure of dispersion of the weights of 
pupils would indicate to him how far above the average he 
should go before a safe board was obtained. Students who are 
immersed in the study of statistical methods, learning for the 
first time in a formal way the advantages of averages and the 
many things that can be done with them, sometimes require a 
few moments to comprehend the element of humor when the 
story is told of the young engineer who was building a bridge 
over a river and ascertained the average height of the masts 
and towers of the boats which plied the river, in order that he 
might know how high above the river to build his bridge. 

There are several measures of dispersion common in statis- 
tics. These are, the standard deviation, the mean deviation, the 
quartile deviation, and the range. The last is merely the dif- 
ference between the extremes of the series; it is not extensively 
used because the extreme values are very unstable. The mean 
deviation is the average distance out from the average (usu- 
ally the mean) that the measures lie. The quartile deviation is 
the average distance out from the median that half of the 
measures on either side of the median lie. The standard devia- 
tion is a more complex function, being calculated from squares 
of deviations. 

Usually students can readily understand averages, because 
they are accustomed to them. The measures of dispersion— 
sometimes called measures of variability—are usually new, 
and students are curious about them. Although measures of 
dispersion are significant for characterizing series, they find a 
larger use in the employ of more complex formulas. They have 
another important use. When one is comparing the averages of 
two or more distributions, he cannot well tell how significant 
the difference is unless he knows how much the distributions 
spread. A difference of ten between averages might be very 
small, if the two distributions spread over a range of several 
hundred; or it might be very large if each distribution spread 
over decimal divisions of a single point. 
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Measures of relationship. Correlation coefficients are com- 
mon in theses and in many other studies. Though a considerable 
amount of calculation may be involved in determining the 
degree of correlation, the concept is essentially simple. Corre- 
lation is the extent to which an individual tends to be in the 
same relative position in each of two series. That is, if two 
tests were given to a class of pupils, the correlation between 
these two sets of test scores would be high if the pupil who 
was at the top in one set of scores was at or near the top of 
the other, and so on down through the list, the pupil at the 
bottom of one group of scores being at or near the bottom of 
the other series of scores. Perfect correlation, denoted by the 
figure 1.00, means that there is exact correspondence. No cor- 
relation at all, denoted by .co, means that, if you know what 
a pupil’s score on one test is, you have no basis for estimating 
his score on the other test. Negative coefficients indicate a rela- 
tionship between the two series, but an inverse one. For 
example, a correlation value of —r.00 indicates that the cor- 
respondence is perfect, but in inverse order, the pupil at the 
top of one series being at the bottom of the other, and so on 
through the series. 

Of course, correlation is not confined to test scores ; these are 
used merely as an illustration. Correlation can be calculated 
between any two sets of. paired measures—that is, where a 
measure in one series is connected with a particular measure in 
the other series, such as the two test scores of a single pupil. 
These scores are two quantitative representations of the same 
case; if there is not this connecting link, one cannot calculate 
correlation. One cannot take any two series of measures and 
find the correlation between them, since correlation cannot 
exist unless the measures are definitely paired. 

Tt was pointed out in the preceding chapter that correlation 
indicates that the two series of values have some elements in 
common. The inference that one variable directly causes fluctu- 
ations in the other, however, is not immediate and must be 
made only after a careful inspection of the logical connection 
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between the two series. A common use of correlation is for pre- 
diction; the higher the degree of correlation, the greater the 
accuracy with which one can predict a score or other per- 
formance in a second series of measures, when the value in the 
first series is known. 

The interpretation of the significance of various degrees of 
correlation is a matter requiring some background in the field 
of application and some understanding of the mathematics 
underlying the formula. There is no immediately simple inter- 
pretation that is safe. Correlations between school subjects 
(ie., between the standing or ability of pupils in two different 
school subjects) usually run from about .40 to .60 or perhaps 
slightly higher. Two different forms of a standard test given 
to the same group of pupils should, ideally, correlate perfectly 
(1.00); they are more likely to correlate from .60 to .go. 
Probably for general purposes the best method of interpreting 
a given correlation coefficient is to project it against the back- 
ground of such general knowledge about the usual range of 
coefficients in a particular field. For further discussion of 
interpretation the reader is referred back to the treatment in 
the preceding chapter, and particularly to the references in 
footnotes 50-58. 

Correlation has an extensive use in connection with the 
critical study of tests and other instruments. The correlation 
of two series of measures that are supposed to represent the 
same thing (such as two applications of a standard test, or of 
comparable forms of it), is known as the coefficient of re- 
liability. If it is low, it indicates that the instrument does not 
succeed in eliciting the same kind of performance from one 
time to another. Whether this is because the instrument is 
faulty, or because the performance has actually changed, can- 
not be told. The correlation of test results with a criterion is 
known as the coefficient of validity. These uses are treated 
fully in statistics and measurement texts.” 


7 Henry E. Garrett, Statistics in Psychoiogy and Education, Chapter VI, “Some Appli- 
cations of Statistical Method and Techniques to Tests and Test Results,” pp. 266-301. 
New York: Longmans, Green and Co., 1926. 


608 EDUCATIONAL RESEARCH 


Error and probable error. One does not work with figures 
long before he comes to realize that there are all kinds of 
approximations used, and that there are all kinds of errors 
involved. The statistician’s idea of error, however, is very dif- 
ferent from what the average person thinks of as an error. 
People in general think of an error as a mistake; the statisti- 
cian’s conception of it is only as a departure from truth, and 
he usually thinks of it as a relatively chance departure. These 
chance departures from an idealized concept of truth come 
about through the operation of many small factors which, 
when taken together and in large numbers, are regarded as 
chance. Truth, to the statistician, is usually a sort of an aver- 
age, the value one would get most often if he kept on measur- 
ing or trying something for an indefinitely long time. 

A simple illustration may be found in any of the devices 
that operate by chance. Tossing a coin should, ideally, give an 
equal number of heads and tails. Practically, one knows that 
if he tosses a coin four times he is not likely to get two heads 
and two tails; what he gets, however, other than two heads 
and two tails, is regarded by the statistician as being just that 
much in error, since the true values (obtained by tossing an 
ideal coin many times) are 2 and 2. It will readily be seen from 
this illustration that the statistician does not mean that one 
has measured something wrong or made any mistakes; his 
numbers may be accurate so far as the objective events are 
concerned, but the statistician knows that events vary, and he 
is interested in the values they will take in the long run. Any- 
thing else is to him an error, even though it may be actual.* 

To apply the concept to test scores, it is quite possible that 


Charles W. Odell, op. cit., Chapters XI, XIX, XX. 

G. M. Ruch, The Objective or New-Type Examination, Chapters II, III, XV. Chicago: 
Scott, Foresman and Co., 1929. Pp. 478. 

Most books on statistical methods and on tests and measurements discuss problems 
of validity and reliability. 

8 There are of course also errors in the usual or non-statistical sense. See the following: 

Walter S. Monroe, “Dependability and Value of Survey Types of Investigations,” 
School and Society, XXXVIII (October 21, 1933), 517-22. 
~, The Constant and Variable Errors’ of Educational Measurements. Bureau of 
Educational Research Bulletin, No. xs. Urbana, I.: University of Illinois, 1923. Pp. 30. 

‘Charles W. Odell, ob. cit., Chapter XX. See also “Error” in the Index of Topics. 
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an objective test may be scored accurately and yet will not 
present a very good measure of the pupil who took it. As ex- 
plained previously in Chapter VII, many factors enter into the 
performance of a pupil at any particular time; these various 
factors cause the pupil to do certain things at one time that 
he will not do at another, with the result that his score at any 
time may not be his normal or typical score, and hence it is in 
error as representing his true ability. This is a very useful and 
highly important concept for teachers to grasp ; it helps prevent 
the iron-clad and foolishly mechanical interpretations that 
many teachers are inclined to put upon test scores, especially 
when the test is a well-known “standardized” test and has been 
constructed by persons with well-known reputations. Such 
tests may be excellent, but they do not eliminate variability in 
pupil performance. Teachers should invariably look at a test 
score with the idea that there is likely to be, say five or ten 
points, between that particular score and what the pupil 
typically would do. Such a concept may disrupt the thought 
habits of many teachers, in feeling that they can mark a pupil 
with scientific accuracy after having given him a test, but it 
may serve to bring such teachers to a more nearly correct way 
of thinking about educational measurement. 

All of these comments apply with equal or greater force to 
the research worker. One must be very careful not to conclude 
that one pupil is better than another because the first learner 
made a score of 78 on a test and the second made a score of 
73. It is not unlikely, with the usual test, that another testing 
would show the second pupil as good as the first—or perhaps 
much worse. The statistician has a way of estimating how 
likely these changes are. If he can ascribe the changes to chance 
or assume that the changes will follow the same laws as chance, 
then he can use mathematical tables of chance to determine 
how likely a change of any particular amount will be. 

The unit of measurement which the siatistician uses in 
thinking about chance changes from one measurement to an- 
other is the probable error. The concept will not be defined 
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here in statistical terms, other than to say that it represents 
a function of the standard deviation, which was described 
earlier. What it means is, that a second measurement is as 
likely to differ from the first measurement by more than one 
probable error, as it is to differ by less than one probable error. 
In other words, it is the average expected change, or error, 
caused by the operation of the many elements which are respon- 
sible for a variation in value. Of course, when the variation 
from one time to another is systematic with respect to any ob- 
servable factor, then it is no longer regarded as a chance varia- 
tion, and its relation to the important factor becomes a 
subject for study. It is only the small, relatively unobservable 
factors that are gratuitously (but conveniently) regarded as 
“chance.” 

Perhaps the foregoing remarks will suggest to the reader 
that it becomes important at times to determine whether a 
change, or a difference between any two values, is really caused 
by some significant factor or whether it is a caprice of chance. 
One cannot answer such a question with certainty, unless he 
repeats his experiment or keeps up his measurements for an 
indefinitely long time. But he can answer the question in terms 
of probability, without doing more measuring. This is accom- 
plished by applying the statistician’s measuring rod of probable 
error, If the change (or difference) is equal to only one prob- 
able error, then there is nothing much to be said about it; it 
could easily have been caused by the operation of the chance 
factors that make results vary. If chance factors will account 
for all of a difference, then one cannot say that the difference 
shows the influence of any particular factor, such aş an experi- 
mental factor which he may be studying. All he can conclude 
under such circumstances is that the factor he thought was im- 
portant does not, under study, seem to be effective in causing 
a difference between one group or set of observations, and 
another. 

One may ask the question, “What if the difference is equal 
to two probable errors?” Then, the statistician would answer, 
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it is less likely that it is due to the fluctuations of chance 
causes, but it is still quite possible, and in fact it is so likely 
that one had better not draw any important conclusions. In 
fact, it is not until a difference is equal to about four times the 
value of a probable error, that one can begin to say with some 
feeling of (scientific) certainty that a result is typically dif- 
ferent from another; here the probability is only one in about 
three hundred that the observed difference is the result of what 
are called chance factors and not the result of some significant 
factor. Any difference, therefore, that is equal to or greater 
than four times its probable error is commonly called a signifi- 
cant difference.” 

The reader can probably follow the above reasoning without 
understanding how the value of the probable error is calculated. 
For such calculations he is referred to statistics texts. 

The foregoing discussion has treated only general statistical 
devices. There are special procedures adapted to various par- 
ticular problems. For example, McCall * gives an extended 
treatment of statistical analysis of results obtained from ex- 
perimentation, and Monroe and Engelhart ™ present a similar 
treatment. Bixler 12 devotes a chapter to general aspects of 
statistical analysis. 

Brief description of elementary statistical calculations. 
To give students an introductory idea of the processes of 


9 One will appreciate that statistical significance differs from practical significance. 

E. A. Lincoln, “Insignificance of Significant Differences,” Journal of Experimental 
Education, II (March, 1934), 288-90. 

, “Reliability Coefficients Are Stil Unreliable,” Journal of Educational Psy- 
chology, XXIV (March, 1933), 235-6. 

Charles W. Odell, The Interpretation of the Probable Error and the Coefficient of 
Correlation, Bureau of Educational Research Bulletin, No, 32. Urbana, Ill.: University of 
Illinois, 1926, Pp. 49. 

R. W. Tyler, “What Is Statistical Significance?” Educational Research Bulletin, X 
(March 4, 1931), 115-8, 142. Also found in R. W. ‘Tyler, Constructing Achievement 
Tests, pp. 66-70. Columbus, Ohio: Bureau of Educational Research, Ohio State Uni- 
versity, 1934. Pp. 102. 

10 William A. McCall, How to Experiment in Education, Chapters VI-VIII. New York: 
The Macmillan Co., 1923. Pp. 281. 

11 Walter S. Monroe and Max D. Engelhart, Experimental Research in Education, 
Chapter III, “The Interpretation of Differences in Gains,” pp. 59-76. Bureau of Educd- 
tional Research Bulletin, No. 48. Urbana, Iil.: University of Illinois, 1930. Pp. 105. 

12H. H. Bixler, Check Lists for Educational Research, Chapter V, “Statistical Meth- 
ods Used in Research,” pp. 52-62. New York: Teachers College, Columbia Univ sity, 
1028. Pp, 118. 
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statistical calculations, a number of the commoner ones are 
described herewith. In presenting these verbal directions, the 
authors have no thought that students will “learn statistics” 
from reading them; it is rather the purpose that these may 
furnish a reference list if the instructor wishes to discuss these 
processes. Also, the descriptions may enable one to look more 
expeditiously for what he is seeking in statistics texts, which 
he should consult for fuller treatments. It is not expected that 
the student will understand the following directions without 
explanation and illustration 

These procedures are put in words to avoid a long descrip- 
tion of symbols. The symbol N, which is universal in statistics, 
indicates the number of cases in a series (i.e., in one’s study). 
Ungrouped data means data that are in the form in which one 
gathers them; grouped data means data which have been 
tabulated into a frequency distribution. 

1. Mean. Ungrouped data: Add the values and divide by W. 

Grouped data: Add to the midpoint of any selected interval 
a “correction” which is obtained by multiplying the frequency 
in each interval by the number of intervals it is away from the 

_ selected interval (using negative numbers for intervals below 
the selected one), getting the net sum of these products, 
and dividing by W. Multiply this sum by the width of the 
interval and add or subtract the “correction” according to its 
sign. 

2. Median. Ungrouped data. If N is odd, take the value of 
the (NV + 1)/2 measure; if NV is even, average the N/2 and 
(N + 2)/2 measures. 

Grouped data. Interpolate between the limits of the interval 
in which the V//2 measure lies, assuming that the frequencies 
are evenly distributed over the interval. 

3. Mode (approximations). Ungrouped data. Take the most 
frequently occurring value. 

Grouped data. Use midpoint of the interval containing the 
greatest frequency. 

4 Percentile rank. Ungrouped data. Subtract 74 from the 
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rank of a case (when ranked with “one” at the low end), and 
multiply by 1oo/N. 

Grouped data. Not normally desired. 

5. Percentile points. Ungrouped data. Multiply N by given 
per cent, round product to nearest whole number, and count 
upward from bottom of series (arranged in order of magni- 
tude) till the value having this rank is reached. 

Grouped data. Multiply V by the given per cent; count 
upward in the frequency distribution till the measure having 
this ordinal value is reached. This normally requires inter- 
polating within an interval, assuming that the frequencies are 
evenly distributed over the interval. 

6. Mean deviation. Ungrouped data. Calculate the mean, 
subtract it from each value in the series; average the resulting 
differences, disregarding their sign. (The median may be used 
instead of the mean.) 

Grouped data. The method is involved. See a statistics text. 

7. Quartile deviation. Calculate the 25th and the 75th per- 
centile points; subtract the former from the latter and divide 
the difference by 2. : 

8. Standard deviation. Ungrouped data (first method). 
Square each measure, and sum; divide the total by V; sub- 
tract from this quotient the square of the mean, and take the 
square root of the difference. 

(Second method). Subtract the mean from each value, and 
square the remainder. Sum these squares and divide by N. 
Take the square root. 

Grouped data. Proceed as in calculating the mean for 
grouped data; next, multiply each of the products again by 
the number of intervals it is away from the selected interval, 
(which will make all signs positive); sum these values and 
divide by N. Subtract from this quotient the square of the 
“correction” obtained for calculating the mean (without multi- 
plying by width of interval), take the square root of the 
remainder, and multiply it by the width of an interval. 

9. Correlation (product-moment). Ungrouped data. Arrange 
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logical study. Warren and Mendenhall ** published the first 
description for general use of the method of calculating cor- 
relations on the machines, and of securing powers. A second 
publication, by Warren,’* deals with the fitting of trend lines 
by the least squares method. Wallace and Snedecor" have 
prepared an amplified treatment of the use of the machines for 
correlation, including multiple correlation and partial correla- 
tion. They also include a chapter on coding. The most extended 
treatment of the applications of these machines is a recent book 
edited by Baehne ** and written by men who have had con- 
siderable experience with practical uses of the machines. It 
includes problems of coding as well as other practical topics. 

It was pointed out in the discussion of questionnaires, in 
Chapter VII, that one should consider the possibility of using 
such machines early in his study and plan his instruments and 
early recordings in the light of such use. For example, many 
questionnaires are now pre-coded; that is, various responses 
are given a code directly on the blank when it is printed, so 
that the operator can punch cards from the blank without an 
intermediate step of coding. It might be said that the United 
States Bureau of the Census has for many years employed 
these machines for the tabulating of census returns. Before 
these machines were invented, census returns were not available 
until years after the data were gathered. 

A method of coding has been devised which greatly increases 
the capacity of the cards and makes them practically limitless.1® 

15 Richard Warren and Robert M. Mendenhall, The Mendenhall-Warren-Hollerith Cor- 
relation Method, Columbia University Statistical Bureau, Document No. 1. New York: 
Columbia University, 1929. Pp. 43. 

16 Richard Warren, Hollerith Machine Computation of Least Squares Trend Lines. 
Columbia University Statistical Bureau, Document No. 2. New York: Columbia Uni- 
versity, 1933. Pp. 35. 

17H. A. Wallace and George W. Snedecor, Correlation and Machine Calculation. 
Towa State College of Agriculture and Mechanic Arts, Official Publication, Vol. XXX, 
No. 4, June, 1931. Ames, Iowa: Iowa State College. Pp. 71. Includes a bibliography, 
PP. 70-71. 

is G. W. Bachne, ed., Practical Applications of the Punched Card Method in Colleges 
and Universities. New York: Columbia University Press, 1935. Pp. 442. 

1 Halbert L. Dunn, “Adaptation of New Geometric Code to Multiple Punching in 
Mechanical Tabulation,” Journal of the American Statistical Association, XXVII 
(September, 1932), 279-86. 


Elmer B. Royer and Herbert Toops, “The Statistics of Geometrically Coded Scores,” 
Journal cf the American Statistical Association, XXVIII (June, 1933), 192-8. 
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For current literature, see the topic, “Tabulating Machines,” 
in the Education Index. 


INTERPRETATION 


The past few paragraphs of the section on “Error and Prob- 
able Error” dealt with a certain aspect of interpretation, 
namely, the legitimacy of ascribing an observed difference to a 
factor which varies (changes in magnitude or kind) from one 
observation or set of observations to another. The recognition 
of variations ascribable to chance places a very important 
limitation upon conclusions which one is likely to deduce at 
first glance. The trained research worker knows that differences 
which are observed must first be compared with the conclusions 
that one could draw simply by tossing coins or dice and not- 
ing the variations that laws of chance produce. If a difference 
observed in one’s study is greater than these chance differences 
are likely to be, one can say that the difference is significant ; 
otherwise, it may not be a typical difference, just as the differ- 
ence between two successive throws of dice is not typically 5, 
though one may actually observe such a difference at one 
time or another. 

The process of interpretation is essentially one of stating 
what the results (findings) show. What do they mean? What 
is their significance? What is the answer to the original prob- 
lem? This process calls for a critical examination of the results 
of one’s analysis in the light of his previous analyses concerning 
the gathering of data. That is, all of the limitations of his data- 
gathering must enter into and become a part of his conclusions, 
which grow out of his interpretations of the results. This step 
is as important as any other in the research study. It is almost 
purely subjective, and many errors are made at this point. If, 
however, one is careful, and critical of his own thinking, he 
should be able to make satisfactory interpretations. 

It may be remarked here that the interpretative phases of 
research constitute one of the important reasons why it is 
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necessary for a research worker to know his field of work and 
not depend solely upon his knowledge of investigational tech- 
niques. There are many phases of research which cannot be 
well done unless one is familiar with the field in which he is 
working, in addition to having an understanding of the proc- 
esses of research. One should not, therefore, undertake any 
large or important study in a field until he has had an oppor- 
tunity to obtain background and insight in that field—other- 
wise he cannot help making a rather mechanical and probably 
misleading study. It is wholly erroneous to assume that an 
adequate knowledge of research processes will safeguard one’s 
findings and conclusions. Familiarity with the field of work is 
equally, if not more, important than technical knowledge. Both, 
however, are highly desirable. 

Misinterpretation due to unstudied factors. It may be 
worth while to illustrate a few of the difficulties that arise in 
interpretation. Many of the fallacious conclusions drawn in 
research grow out of the limitations of the studies which are 
made. A given result is composed of many factors; it is not 
produced simply by a single factor. It is true that one may 
know of only one important factor, but this does not justify 
the drawing of immediate conclusions, unless one has taken 
pains to explore other areas and see that other factors which 
might be important have not operated to produce the result. 
For example, a worker may conclude that a certain method of 
teaching spelling is better than others he has studied. In real- 
ity (if he has not studied or controlled other factors), the 
superiority of results which he finds for one group of pupils 
taught by a certain method may be due to a higher average 
intelligence, an older age, a better teacher, a better previous 
teacher, a better set of readers, a better previous (or present) 
reading teacher or method, better home conditions with more 
emphasis upon spelling, a greater incentive to learn, more time 
to study, more sleep at night, etc. The factors which condition 
any result are innumerable; usually we assume that those we 
have not studied will average out and will be the same for one 
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group as for the other. This is an assumption which is con- 
venient, but not always safe. 

Not long ago a study reported a correlation of .50 between 
scholarship and teachers’ estimates of their pupils’ intelligence, 
and a correlation of .75 between scholarship and character 
ratings. One may conclude that character factors are more im- 
portant in producing scholarship than is intelligence. But there 
are other possible interpretations. Scholarship was rated by the 
teachers, and the higher correlation with character ratings may 
mean merely that scholarship and character are not entirely 
separated in the minds of the teachers, so that those pupils who 
“behaved” tended to get better marks, and those pupils who 
were regarded as generally undesirable received lower marks, 
etc. Before one can draw conclusions with reference to the 
relative strength of the relation between intelligence and scholar- 
ship and character, he must study carefully the effect of the 
factor introduced by having the same person judge all three 
characteristics. 

In some instances the interpretation is difficult or incon- 
sistent with other results because one has conceived of his 
problem in too narrow terms. McGeoch * calls attention to this 
difficulty in a review of the studies on the whole-part problem 
in learning. The fallacy in the assumption that the study can 
be concerned with proving that one method is better than an- 
other is well stated in her own words: 


Such questions imply that there is a method of learning which is 
really or inherently superior, and that certain experimental conditions 
permit this superiority to show itself, while other conditions mask or 
hide this intrinsic superiority. McGeoch °! has called attention to the 
fact that such a point of view is contrary to the logic of science. No 
phenomenon known to science is unconditioned: an absolutely best 
method of learning has no empirical existence. The scientific goal, and 
likewise the goal of the educator, is to discover by experimentation all 
the possible factors of which the efficiency of any given learning 


20 Grace O. McGeoch, “A Revaluation of the Whole-Part Problem in Learning,” 
Journal of Educational Research, XXVI (September, 1932), 4. 

21J. A. McGeoch, “The Acquisition of Skill,” Psychological Bulletin, XXVIII (June, 
1931), 413-66. 
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method is a function. The question is not: “Is the whole method 
superior to the part method?” The question is: “Under what condi- 
tions is the whole method superior to the part method?” 

One suspects that many of the contradictory findings which 
abound in educational research are attributable to an inade- 
quate conception of the problem to be studied, with a con- 
sequent lack of attention to variables which are really 
significant. These, and other illustrations which might be added, 
will aid in making clear the importance of the early choice 
of factors for study. One’s conclusions are always limited to 
the factors studied, and the cautious person will not draw gen- 
eralizations which involve factors and conditions which he has 
only assumed. Several articles have been written calling definite 
attention to assumptions that are made in different fields of 
work.,** As an example of very cautious conclusions, the reader 
is referred to a study by Easley; ? even though he may not be 
able to follow the statistical part of the report, he may note 
the quality of thought which is involved. 

ignoring selective factors, Although selective factors are 
in reality a special case of the foregoing topic, they are of 
sufficient importance to warrant specific treatment. They are 
frequently ignored and lead to unwarranted conclusions. As an 
illustration, one may study the per cent of pupils who fail in 
the different years of high school. He will likely find that the 
per cent is high in the first year and low in the last year. He 
might, therefore, conclude that the first year of high school is 
hard and the fourth easy. We know, however, that the observed 
result is due to selection—the pupils who fail in the first year 
do not go on in any large numbers into the fourth year. A re- 
search worker who did not know this would be as likely to 


22 Gordon Hendrickson, “Some Assumptions Involved in Personality Measurement,” 
Journal of Experimental Education, II (March, 1934), 243-0. 4 

Douglas E. Scates, “Types of Assumptions in Educational Research,” Journal of 
Educational Psychology, XXVI (May, 1935), 350-66. 

Ralph W, Tyler, ‘‘Assumptions Involved in Achievement-Test Construction,” Educational 
Research Bulletin, XII (February 8, 1933), 29-36. Also reprinted in his Constructing 
Achievement Tests, pp. 5866. Columbus, Ohio: Bureau of Educational Research, Ohio 
State University, 1934. Pp, r02. 

23 Howard Easky, “An Attempt to Isolate the Factor of 
Journal of Psychology, XLIII (April, 1931), 202-15. 
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draw the wrong conclusion as the right one; there is nothing 
in the figures themselves to tell him. 

One may say that nobody would overlook such a selective 
factor; but that depends entirely upon one’s familiarity with 
the field of work. To take an illustration not so familiar 
to educators, it has been observed that cancer is less prevalent 
among prisoners than among the population at large. One can 
calculate the diet of the nation on the basis of food produced 
and find that the rations in prisons contain less meat than the 
average. If one has not developed research (or educational) 
cautions, he is likely to be swayed by the implication. Even 
students of the problem have been. But some one who has a 
greater understanding of the cancer problem is suspicious of 
such conclusions and looks up the average age of prisoners. He 
finds it is below thirty-five, and cancer is known to be rare 
below this age. The conclusion that diet is responsible for the 
low incidence of cancer among prisoners therefore becomes 
less convincing. To establish a causal relationship between meat 
and cancer would require observing the effects of the given diet 
upon normal age distribution. If such further study showed a 
relationship, it would need careful testing by various kinds of 
attack before the conclusion could be regarded as valid. 

Not many years ago it was generally believed that delinquent 
children were of lower than average intelligence. Facts proved 
it. One had only to measure the intelligence of those children 
who were in various penal institutions to find plenty of evidence 
to this effect. Then it began to be realized that not all delin- 
quent children were in institutions, and that the more intelli- 
gent ones, on the whole, escaped apprehension. A very definite 
selective factor (intelligence) operated to make the children in 
institutions appear to have Jess than average intelligence, but 
one does not have any right to generalize such findings to in- 
clude all children who are delinquent in the sense of doing 
wrong. Selective factors are indeed subtle and pernicious. 

The statement seems warranted that all studies concerned 
with the inheritance of intelligence and of personality char- 
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acteristics, as well as special talents, that were made previous 
to the important nature-nurture study,** ignored very definite 
selective factors, and from the standpoint of scientific proof 
their conclusions concerning the strength of inherited charac 
teristics suffer in the same way that the conclusions in the 
preceding illustrations do. Although such studies—frequently 
reported in books on educational psychology and principles of 
education—show that such successions of characteristics do 
occur in families, they are of little significance scientifically, 
notwithstanding their widespread acclaim. The very strong 
factors of home environment, and of living in a general socio: 
economic class, are utterly ignored.*® 

Difficulties of interpretative evaluation. The two preced- 
ing sections have been primarily concerned with interpreta- 
tions connected with causal relationships. The chief difficulty 
of interpretation connected with studies of the simple descrip- 
tive type, such as normative-survey studies, is in relation to 
evaluation. One is called upon, either to leave the significance 
of his findings in doubt, or to place some practical interpreta- 
tion upon them which involves evaluation. For example, there 
is the problem of evaluating opinion. If a questionnaire study 
shows that three-fourths of the high-school teachers of a city 
consider the curriculum, as they are teaching it, very closely 
related to the needs of people in general community living, the 
question is still open as to whether this articulation with non- 
academic life is a fact, or whether the teachers are in reality 
naive and know so much of school and so little of everyday 
life that they lack sound judgment on such a question. What 
the result does show is that the teachers think that way; any 
further interpretation is done with considerable hazard. Atten- 
tion was earlier directed to the difference between objective 
facts and facts of opinion. Even a roo per cent vote on some- 


24L. M. Terman and Others, Nature and Nurture: Their In 
and Achievement, Parts I and II. Twenty-seven: 
for the Study of Education. Bloomington, TIl.: 
Pp. v + 460, xv + 353. 

25 See for example, Barkey S. Sanders, Enviro: 
Warwick and York. 1934. Pp. 375. 
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thing does not necessarily mean that it is an objective fact; it 
only means all of a certain group think that way. 

Another difficulty grows out of interpreting frequency in 
some of the normative-survey studies, particularly those deai- 
ing with an analysis of the frequency with which certain topics 
or names occur in adult reading, and the like. There may be 
little significance in such frequencies that is undebatable. Does 
the frequent mention of a name in the newspaper mean that 
the given name is more important or more worthy of attention, 
than less frequently mentioned names? Any one familiar with 
newspapers knows that there are in any city many citizens 
whose acquaintance and counsel are worth seeking as much 
as those whose names are much oftener mentioned, and the 
same thing is true of national life. Does one not need to know 
many things as means of understanding current events, when 
it is reasonably certain that these antecedents will not have an 
appropriate frequency of mention in current reading material? 
Can cruciality be inferred from frequency, or can the impor- 
tance of an event for a single individual be inferred from the 
prevalence of the event in the total group? 

Another kind of interpretation grows out of the evaluation 
which one makes of the findings. This is not always done, but 
it is likely to come in as an element which colors many con- 
clusions. For example, if one looks at a table showing average 
salaries of teachers throughout the nation and finds that 
kindergarten teachers receive on the average about the same 
as elementary teachers, is this a fact that is fortunate or unfor- 
tunate? Should it be, or does it represent something that is 
wrong in some respect? Suppose, further, one should find the 
average salaries for elementary and high-school teachers con- 
stantly approaching one another over a period of years, is 
this cause for alarm? Or does it indicate a gradual trend 
toward a wholesome situation? One will at once recognize, in 
the answers he tends to give to such questions, the influence 
of his ideas on the subject. 

Although most workers will refrain from making interpre- 
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tations which are patently personal, probably most of us can- 
not always draw the line between our personal philosophies 
and a simple statement of facts. Our thinking is so colored 
by our intrenched attitudes that it is very difficult to separate 
the two. Further, it is not here argued that all research should 
be mechanically colorless; one should make an attempt to 
separate conclusions which represent a considerable amount of 
personal belief, from the relatively incontrovertible logic of his 
facts, but he does not always have to omit his somewhat per- 
sonal interpretations. He should merely state them in such a 
way that they do not mislead the unwary into thinking that 
“statistics prove” certain things that the writer thinks. 

There is yet another type of interpretation which is often 
worth while, and that is the explanation of the findings in 
terms of their practical implications. What do they mean for 
society? What can be done about them? What possibilities 
do they open up? How do they change the picture in general ? 
It will be recognized that these fall mostly under the head- 
ing of the personal interpretations mentioned in the last para- 
graph. They may be very worth-while conclusions; in fact they 
may be the most important part of the research, so far as 
getting anything done is concerned. Such interpretations are 
perfectly legitimate, but they should, as previously stated, be 
put in such a way that the uncautious reader will not confuse 
them with the factual interpretations. Such recommendations, 
constitute a part of the general interpretation which the re- 
search worker should undertake, if he is interested in making 
his research practically helpful. 

Other discussions of interpretation in this text. Inasmuch 
as interpretation varies considerably with the type of research 
which is in process, it has seemed appropriate to inject dis- 
cussions of interpretation at various points of the text where 
the methods were being presented. For example, in Chapter 
VI, problems of interpretation were discussed in connection 
with historical material. It is necessary for one to subject his 
material to careful scrutiny and to examine it for internal con: 
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sistency, external consistency, and likelihood of representing 
misleading factors, before he presents it as representative of 
a certain period and place. There are also large problems of 
historical interpretation which represent a historical philoso- 
phy, relating to the relative importance of basic factors in 
causing changes. 

In Chapter VII, interpretation was discussed in connection 
with survey testing, and the importance of recognizing many 
factors that might account for the results was stressed. In 
drawing conclusions from questionnaire studies, and from doc- 
umentary frequency analyses, it was stressed that inferences 
relating to cruciality and social importance were generally re- 
quired; also, that a basic philosophy, or pattern of thought 
into which the facts might fit, was necessary before such 
studies attained their maximum usefulness. 

Chapter VIII, dealing with the interview, observation, and 
appraisal procedures, called attention to conditions that must 
be satisfied before the facts obtained could be regarded as 
facts and consequent interpretations drawn. It was pointed 
out that judgments are not objective facts, but they may 
nevertheless be important facts, when properly used and in- 
terpreted. Index numbers, and the results of all other rela- 
tively objective procedures for appraisal purposes, must be 
interpreted in the light of subjective judgments that have 
entered into their construction or use at some earlier stage, 
and which condition the conclusions that are finally drawn. 

In the chapter on experimental work (Chapter IX), the 
difficulties of interpretation when control groups are not used 
were pointed out, and the proper conditions for inferring 
causal connections were described. 

In the preceding chapter, dealing with four methods of re- 
search—the causal-comparative, correlation, case-study, and 
genetic—considerable attention was given to interpretation. 
It was stressed, especially in connection with the section on 
the correlation method, that the inferring of cause is difficult 
and cannot be done mechanically from the objective evidence. 
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Tt was pointed out that two correlated series may not cause 
one another, but that hoth of them may be results of some 
common causal factors. The necessity for comparable data as a 
basis for statistical generalizations from case studies, and the 
need for studying individuals rather than groups in the genetic 
method, were emphasized. 

Further illustrations of difficulties in interpretation and of 
erroneous conclusions will be given in the following chapter. 

Importance of comparison in interpretation. Underlying 
all research is the element of comparison. In fact, whenever 
we speak of variables, and variation, we recognize implicitly 
the comparison of one value with another, at some other place, 
time, or under other conditions. It seems appropriate therefore 
to end the discussion of interpretation with a reference to this 
element of comparison, which is fundamental to all research. 

In some cases, research undertakes to set up a criterion with 
which results can be compared. For example, this is one of the 
outstandingly important elements in experimental research, 
and the control-group type of experimentation is recognized as 
the most crucial type for the study of a single variable, the 
control group of course providing the criterion. In other cases, 
research is concerned chiefly with the production of results, 
which, for practical purposes, must be compared before they 
attain significance. 

Because of the variety of techniques involved in school 
surveys, it seems fitting to quote from Caswell a review of the 
methods of comparison which have been found useful in such 
undertakings: 7° 


1. Comparison of units in the system. The above heading includes 
comparisons of schools, grades, groups of schools, etc., within the 
school system being studied. 

2. Comparison with comparable cities. When data on the school 
system being studied were evaluated by comparison with other cities, 
selected because of their similarity in other characteristics, this was 
classified under the above heading. 


26 Hollis L. Caswell, “Survey Techniques,” Educational Administration and Supervision, 
XIX (September, 1933), 431-41. 
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3. Comparison with neighboring cities. Upor. occasion data were 
evaluated by comparison with cities in the same general territory re- 
gardless of their comparability otherwise. 

4. Comparison with average practice. This includes all cases in 
which the basis of evaluation was comparison with the average of a 
general group or classification. $ 

5. Comparison with outstanding practice. In some cases data were 
evaluated by comparison with situations selected because of their out- 
standing merit. The above heading includes cases of this type. 

6. Equated groups. The experimental procedure ... depends for 
evaluation upon a comparison with respect to a variable factor of 
groups equated with respect to other factors. When such comparison 
was the basis of evaluation, it was included under the above heading. 

7. Comparison with generally accepted standards, This classifica- 
tion includes standards of accrediting agencies, teacher training, etc. 
Only cases in which the standards were objective were included. Cases 
involving individual judgments were included under headings eleven or 
twelve. 

8. Test standards, Since data secured from the application of 
standardized tests depend for interpretation on the standards of the 
tests, these standards were considered the basis of evaluating test data. 

9. Score-card and rating-scale standards. As in the case of test 
standards, data secured from score cards and rating scales depend for 
evaluation upon the standards of these scales. 

10. Research results. Whenever evaluation was made by referring 
to a specific research, this was classified under the above heading. 

zı. Judgment of the survey staff. Whenever the data were declared 
to represent a desirable or undesirable condition and no basis of 
evaluation was given other than the judgment of the survey staff, this 
was classified under the above heading. 

12. Opinion of educational experts. Whenever the opinions of edu- 
cators not on the survey staff were brought to the support of an 
evaluation, this was classified under the above heading. 


Concluding statement. This chapter has been concerned 
with the general aspects of analysis in research. It was pointed 
out that the analysis of data after they have been collected is 
dependent upon various kinds of analyses made at the begin- 
ning of the study. Analysis, in fact, must be pursued through- 
out a research undertaking. General procedures for beginning 
analysis were suggested, such as mechanical manipulations of 
data, and taking a longer perspective on one’s work. 

One section was devoted to certain statistical procedures in 
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analysis. It has not been the purpose of this chapter to go into 
the technical details of statistical work, but rather to describe 
a few of the more important statistical concepts, and to indi- 
cate their general use in carrying on analysis in quantitative 
research. 

The third part of the chapter was devoted to interpretation 
of the findings, after the data had been analyzed. Considerable 
difficulty in interpretation is related to the assignment of 
causes. Conclusions should be limited to the factors actually 
studied, since other causes than those to which one has given 
attention may prove to be more important. Selective factors, as 
causes, are subtle and likely to be overlooked. Another type 
of difficulty in interpretation is related to evaluation. If one’s 
findings are to be of the greatest practical value, he needs often 
to evaluate them; yet he should endeavor to keep such evalua- 
tions distinct from the actual findings. Interpretation is one of 
the most important steps in research and should be undertaken 
with extreme care. 


PROBLEMS AND EXERCISES 


1. Apply the discussion of early analysis to some research study 
(such as those given in any of the preceding five chapters), and illus- 
trate by specific reference to the facts of the study what would be 
included in each of the four things to be analyzed, as pointed out early 
in this chapter. Suggest what considerations might enter into one’s 
reasoning in coming to decisions on these points. 

2, Select an example of each of the major types of research dis- 
cussed in this book, and indicate the statistical procedures employed in 
the analysis and interpretation of the data. Suggest additional methods 
of analysis which might have been used with profit in given studies, 
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CHAPTER XII 


THE FORMULATION OF CONCLUSIONS AND 
GENERALIZATIONS 


The object of research. To secure a better perspective of 
the nature and function of generalizations in educational 
research, the reader should review again the purposes and 
processes of research. The chief purposes for which research 
is conducted are: (1) to determine the status of phenomena, 
past and present; (2) to ascertain the nature, composition, and 
processes that characterize phenomena; (3) to trace growth, 
change, and developmental history ; and (4) to study cause- 
and-effect relationships. The methods of research devised for 
the pursuit of these purposes have been discussed in preceding 
clapters of this book. 

In the search for truth the investigator starts with some 
felt need or difficulty, and employing such knowledge of the 
situation as he may already possess or secure from a direct 
preliminary examination of the object or from previously re- 
ported experiences, he formulates some tentative conclusions, 
hypotheses, or suppositions as to the probable nature of the 
phenomenon under investigation. These hypotheses may be 
employed in the further examination of the data that are 
already available in the form of recollections, documents, 
and critical surveys by previous students of the same problem. 
Using these data, each hypothesis is carefully tested in both 
thought and fact and accepted, modified, or rejected as the 
available evidence indicates. If the evidence already available 
is sufficient to establish the hypothesis or hypotheses under 
scrutiny, the investigation may be concluded at this point, but 
if insufficient, a new appeal to experience must be made. The 
most effective procedures of these new approaches to experience 
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have been discussed in the preceding chapters of this book 
dealing with methods of research. 

Generalizations should be carefully formulated. In an 
earlier chapter (IV), the characteristics of a good hypothesis 
were enumerated as follows: (1) a good hypothesis is con- 
ceivable and in agreement with the observed facts; (2) a good 
hypothesis does not conflict with any law of nature which is 
known to be true; (3) a good hypothesis is stated in the 
simplest possible terms; and (4) a good hypothesis admits 
of the application of deductive reasoning. These same charac- 
teristics are applicable to the conclusions drawn from educa- 
tional investigations and should be so employed in evaluating 
the generalizations reached. 

Educational generalizations should be conceivable and in 
agreement with fact. As the research worker comes to the con- 
clusion of his investigation, he should review carefully the 
evidence for and against each hypothesis. In the first place, 
each generalization must agree with the facts revealed by the 
investigation. The absence of agreement between the conclu- 
sions and the facts is so complete in some studies that the 
collection of data in such instances appears to have been 
nothing more than a formality, influencing to a minimum the 
investigator's already preconceived conception of the phenom- 
enon. After the investigation is completed in the conventional 
sense, all the steps in the process should be checked with great 
care. And the investigator should ask himself repeatedly: “Are 
the data complete?” “Are the data accurate?” “Are the con- 
clusions in agreement with the data?” and “Are the conclusions 
conceivable and in agreement with common sense?” 

This final question suggests that new appeals to experience 
will ordinarily produce only a limited number of new data, 
collected from some particular point of view. Since this is true, 
the investigator will desire to check each generalization care- 
fully, not only against the facts of his own study but against 
the facts and experiences of other investigators. This is a par- 
ticularly important step in all research. We have already seen 
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how all investigations start with facts, go from facts to hy- 
potheses, and from hypotheses again to fact. As one turns 
to facts for the verification of hypotheses, one must return not 
only to the facts of one’s own investigation but again to the 
facts of previous studies. One reason why the conclusions of 
some research workers never appear to be conceivable, even 
though in apparent agreement with the facts brought forward 
in their own investigations, is that they neglect to check their 
generalizations against the wider field of facts made available 
by previous studies. 

In this connection, the reader’s attention is called to four 
fairly common instances in which the conclusions of investi- 
gations do not accord with the known facts. There is, first, the 
case in which the conclusions of the investigator are directly 
contrary to the facts. This situation arises occasionally where 
there are strongly preconceived ideas of the results. This type 
of conclusion probably needs no further explanation. Then, 
there is the case in which the investigator assumes that the 
names given to data-gathering devices accurately indicate what 
they measure. If one assumes, merely because a given data- 
gathering instrument possesses a certain name, that it, without 
further evidence, really measures what it purports to measure, 
one is almost certain to be led sooner or later into erroneous 
conclusions. Many of the generalizations relative to the rela- 
tionship between intelligence and scholastic success, for ex- 
ample, are subject to this error, inasmuch as most of the 
so-called intelligence tests do not really measure intelligence. 
There are many other similar instances in the literature of 
education. A third type of situation, in which generalizations 
may not be in accord with the data, is that in which the facts 
and relationships found to exist under certain conditions are 
held to be true under all conditions. Many illustrations of this 
fallacy may be found in the field of learning, where certain 
facts are discovered about the learning of certain materials 
(nonsense syllables, for example), by pupils of certain ages, 
interests, and levels of intelligence under certain conditions, 
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and these facts are held to be true for all materials, learned 
by all pupils under all conditions. Finally, the generalizations 
made in evaluative investigations are not always in accord 
with the facts, because of the partial measurement of out- 
comes. If one were investigating, for example, the relative 
merits of direct and activity learning, and measured only cer- 
tain outcomes, it would be erroneous to say that the facts 
show either the direct or indirect method of learning superior 
to the other, since only part of the outcomes were studied, 
with only a limited number of pupils studying certain ma- 
terials under given conditions. Experienced research workers 
limit their conclusions to bring them into close agreement with 
the known facts. 

Generalizations should not conflict with the known laws 
of nature or previously established generalizations. A sec- 
ond test of the dependability of the conclusions drawn from 
any particular investigation is their agreement with previously 
established generalizations. This test is important because, if 
a conclusion is in conflict with a generalization already held 
to be true, the disagreement must be resolved, either by show- 
ing that the conflict is not real or that the conflicting statement 
is false. According to the principle of “excluded middle,” a 
statement or its denial must be true. Either a thing is true 
or it is not true. If, of two generalizations, one appears to 
deny the other, one or the other of the two statements cannot 
be true. Thus, before an investigation can be considered com- 
plete, a conflicting conclusion must be shown not true or not in 
disagreement with the generalization reached in seeking to 
interpret the data. 

Educational generalizations should be stated in the sim- 
plest possible terms. Attention has already been directed (in 
Chapter IV) to the fact that the number of hypotheses should 
not be needlessly multiplied. Every generalization should be 
reduced to the simplest possible terms consistent with clarity 
of expression. The dangers arising from too implicit confidence 
in the accuracy of educational terminology were emphasized in 


CONCLUSIONS AND GENERALIZATIONS 635 


a preceding section of this chapter. The terminology must be 
simple and direct but of such a character as to summarize 
as precisely as possible the actual facts of the case. Each gen- 
eralization should be studied with extreme care, in terms of 
accuracy, clarity, and simplicity of expression. (Certain prob- 
lems of terminology are discussed in Chapter XIII.) 

Educational generalizations should be amenable to de- 
ductive reasoning. Remembering that the generalizations of 
one study may become the point of departure in a subsequent 
investigation, they should be formulated in terms which render 
them subject to further verification. Attention has already 
been directed (in Chapter IV) to the futility of employing 
mystical and ethereal forces to explain educational phenom- 
ena. If the projected cause of a phenomenon cannot be verified 
in fact, however true it may appear, it must be discarded in all 
scientific explanations for something that can be substantiated. 
That every generalization be verifiable is a fundamental prin- 
ciple of scientific method, which the investigator should con- 
sider carefully before casting a conclusion in final form. He 
may well ask himself this question with reference to each 
generalization: “Is this conclusion stated in a form that it 
may be further verified by subsequent research?” 

Some common fallacies of reasoning leading to erroneous 
generalizations. In an earlier chapter (I) of this discussion 
eleven fallacies of reasoning were enumerated. These fallacies 
were given as follows: (1) argument from a single or limited 
number of instances; (2) argument from positive instances, to 
the neglect of negative cases ; (3) the omission of evidence con- 
trary to one’s opinion; (4) failure to observe important cir- 
cumstances attending different phenomena; (5) erroneous 
conclusions due to prepossessions, preconceived ideas, and 
prejudices; (6) attributing to a single variable, effects result- 
ing from two or more variables; (7) inaccurate instruinents 
of measurement: dependence upon subjective judgment, esti- 
mates, and guesses; (8) argument from analogy; (9) failure 
to discriminate between material and immaterial circum- 
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stances; (10) mistaken inferences of various sorts; and (11) 
generalization from insufficient data. 

Fowler lists the following fallacies incident to induction: ? 
(2) fallacy of non-observation, consisting of neglect either of 
some of the instances or of some of the circumstances attend- 
ant on a given instance; (2) fallacy of mal-observation; (3) 
errors incidental to classification, nomenclature, and termi- 
nology; (4) errors originating in the employment of the 
inductio per simplicem enumerationem ; (5) errors commonly 
arising out of the use of the inductive methods: (a) mistaking 
a for the cause of b when the real cause is c, (b) mistaking a 
for the sole cause when a and c are the joint causes, (c) mis- 
taking joint effects for cause and effect, (d) mistaking the 
remote cause for the proximate cause, or the reverse, (e) neg- 
lecting to take into account the mutual action of cause and 
effect, (f) inversion of cause and effect; and (6) false analogy 
including argument from antiquity and final causes. As the re- 
search worker comes to the end of his investigation, it is highly 
desirable that he take careful inventory of the data, processes, 
and conclusions of the study as a whole by literally checking it 
for each of the foregoing fallacies. 

No attempt will be made to elaborate upon each of the fore- 
going fallacies separately, but several deserve special mention 
because of their common violation in educational research. 
Much has already been said in the literature of education about 
the dangers involved in generalizations arising from the study 
of too few cases. It is probably unnecessary to say more 
here. The argument from positive cases and the neglect of 
negative cases is a fallacy more frequently found in reasoning 
from incidental observation than from systematic observation. 
Although too much emphasis probably cannot be placed upon 
the dangers lurking in the shadow of preconceived ideas, per- 
sonal prejudice, and mental inflexibility, we shall not stress 
this point further. The importance of baving accurate data 

1 Thomas Fowler, The Elements of Inductive Logic. (Third Edition, corrected and 


revised.) Oxford: Clarendon Press, 1876. Pp. xxvii + 360. 
2 Ibid., pp. 250-351. 
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with which to reason has also been emphasized in numerous 
places in this volume. Special attention was called to the dan- 
gers of non-observation in the application of principles of 
agreement in Chapter X. Attention has also been directed in a 
preceding section of this chapter to the fallacy of accepting 
the names of objects or instruments as accurately descriptive 
or as guarantees of successful performance, for example, intel- 
ligence tests as measures of native intelligence. An equally 
misleading problem arises in the classification of data. The 
assignment of data to categories should be studied carefully, 
since it is likely to be a highly subjective problem and both 
the investigator and the reader may be misled by the nomen- 
clature employed. Although the preceding aspects will not be 
further emphasized in this discussion, it seems worth while 
to analyze and illustrate in greater detail some of the more 
common fallacies arising out of the use of inductive methods. 

Some common fallacies of induction. Fowler’s previously 
cited list of six common fallacies of induction includes those 
of mistaking @ for the cause of 6 when the real cause is c, of 
mistaking a for the sole cause when a and c are joint causes, 
and of mistaking joint effects for cause and effect. The illus- 
trations which follow in the next paragraph illustrate rather 
clearly the error of mistaking a for the cause of b when the real 
cause is c, although it is possible in some of the examples to 
see applications of the other two fallacies. 

In reaching conclusions, a common mistake for inexperi- 
enced investigators, in the case of two or more attendant 
circumstances or results, is to interpret one factor as causal, 
overlooking some common or third factor with a causal rela- 
tionship to one or more of the several results involved. For 
example, it was found that the greater the amount of Latin 
taken in high school the better the college marks of the stu- 
dents in question. This fact was used as a basis for a recom- 
mendation that one should take Latin in high school in order 
to succeed at the college level, overlooking the causal factor of 
intelligence which operated both in the election of Latin in the 
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secondary school and in college performance. An investigation 
of the composition of high-school boards revealed that urban 
secondary schools ranked high with accrediting organizations 
and had boards made up, in most instances, of profcssiona} 
men, while rural schools ranked lower in terms of accrediting 
ratings and had boards composed of farmers. The conclusion. 
reached was that improvement of standards in rural schools 
could be brought about by placing professional men on the 
boards, overlooking the economic and social status of the 
community as a causal factor in determining both school 
standards and the composition of the population and board. A 
study of one hundred eminent Negroes revealed that most 
of them had attended segregated Negro elementary schools and 
ended with the conclusion that such training was a factor in 
producing eminence, not recognizing the fact that most of 
these mature individuals had attended elementary schools 
in the South, with only a few cases who had attended school in 
other parts of the country; of course, during the latter part 
of the nineteenth century when these persons were in elemen- 
tary school, segregated schools were the rule. One investigator 
found that where a dean of girls was present in the high school, 
there was a fairly extensive program of social and extra-class 
activities for girls. The conclusion reached was that the dean 
of girls was responsible for initiation and stimulstion of such 
pupil activities, overlooking the type of community as a causal 
factor in creating a demand for this type of training and school 
officer, as well as overlooking the part played by principal 
or superintendent in promoting pupil activities and possibly in 
seeing the need for a dean of girls,* 

The second type of mistaken inference, namely, that of 
mistaking a for the sole cause of b when a and c are the joint 
causes, may be due to either the fact that both contribute to 
the total effect or that both are essential to the production 
of the result. These situations have already been referred to in 


3 Carter V. Good, “Cautions to the Beginner in Educational Research,” Journal oj 
Educational Research, XXVI (December, 1932), 302-4. eigenen i! 
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the discussion of principles of agreement (Chapter X), and 
in the preceding examples of school boards and the dean of 
girls. A familiar instance of the error appears in many predic- 
tion studies where intelligence (or whatever is measured by the 
so-called intelligence test) is employed to predict scholastic 
success when, as a matter of fact, scholastic success is not 
caused by intelligence alone but by many factors, such as 
interest, effort, methods of work, health, and the ability to get 
along with other people. When a number of factors operate 
to produce a given effect, it is plain that working together 
they may modify, counteract, or even frustrate each other’s 
influence in such a manner that it is exceedingly dangerous to 
generalize about any one factor. A splendid illustration of a 
situation where two or more factors are essential to produce 
a result may be found in the experimental studies of verbal 
learning. Under certain conditions, verbal learning may be 
an ineffective method, but when pursued by pupils with certain 
training, experience, and intelligence, it may become an ex- 
ceedingly effective one. Again, it is dangerous to generalize 
about one cause when the effects of other causes are not 
known. 

The third type of fallacious inference arises out of mistaking 
joint effects for cause and effect. A good illustration of this 
type of error is found in some discussions of reading, where 
certain eye-movements are considered the cause of poor read- 
ing. Without saying that defective eye-movement habits may 
not cause poor reading, there are, however, many instances in 
which the eye-movements observed are merely the symptoms 
of some more fundamental difficulty. Interestingly enough, in 
spite of the many discussions of eye-movements in reading, 
many persons have not grasped the true import of these in- 
vestigations. Studies of the eye in reading may be important, 
but their primary purpose is not to study the eyes as such, but 
to use the knowledge of eye-movements as an index to impor- 
tant mental processes that characterize reading. 

The fourth type of erroneous inference arises out of mistak- 
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ing the remote cause for the proximate cause and the reverse. 
Splendid’ illustrations of this fallacy are found in the con 
ventional methods of discipline. When a pupil appears not ta 
behave as he should, one corrective device is to move the 
pupil to a front seat, or to move other pupils from his vicinity, 
or to remove the pupil from activities where his behavior has 
been particularly unsatisfactory. Obviously all such adjust- 
ments are temporary in character, and a change in any one of 
them will produce merely a temporary effect, the real cause 
of the disturbance or misbehavior being more fundamental in 
character. 

A fifth type of mistaken inference is that which arises out 
of the failure to take into consideration the mutual action of 
cause and effect. Illustrations of this type of fallacy are found 
in the operation of the law of effect, As each subject observes 
the effect of each factor in learning, knowledge of this effect 
(and his attitude toward it) will produce its own effect, and 
henceforth, the operation of the factor, the subject’s knowl- 
edge of the effect, and his attitude, all operate to produce a 
new effect. The new effect may be either positive or negative. 
The law of diminishing returns, in the application of many 
educational devices, also appears to operate in somewhat the 
same fashion. 

The last type of mistaken induction to which Fowler calls 
attention is the inversion of cause and effect. This type of 
error is most common in historical research where cause and 
effect are frequently confused. It is always a question as to 
whether the man produces the times or whether the times 
make the man. Undoubtedly, in many instances the influences 
are of a reciprocal character. One observes at the present time 
certain tendencies toward organization among professional 
workers. Are these organization tendencies the cause or the 
effect of higher standards in training, better salaries, more 
secure tenure? It is sometimes difficult, particularly in uncon- 
trolled observation, to ascertain which is the cause and which 
is the effect, 
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Aside from the above list of fallacies, the investigator 
should not lose sight of the fact that what one sees in the 
study of educational problems depends upon what he knows, 
and that one’s conclusions are accurate and fruitful exactly in 
proportion to the thoroughness and correctness of one’s pre- 
vious knowledge. When previous knowledge is vague and inde- 
terminate, the observation will lack definiteness and depth, 
and will ordinarily lead to unfruitful results. The fact that 
one sees only what he knows how to seek out, taken together 
with the blinding effect of bias, preconceived ideas, and preju- 
dices, makes it almost certain that one rarely arrives at true 
pictures of the objects of educational research. Generalizations 
cannot be made with too great care. 

Certain rules of scientific thinking which are an amplifica- 
tion and restatement of points made earlier in this chapter, 
with respect to formulation of conclusions and generalizations, 
have been well stated by Trow: * 


Give first place to facts. 

Don’t multiply theories unnecessarily. 

Avoid reification of abstractions. 

Regard analogy as a source of suggestions, not as proof. 
Apply the test of agreement. 

Scrutinize all statements purporting to be true. 

Avoid basing conclusions on too few cases. 

Be consistent and coherent. 

Use precise terms. 


ROT. CO SS): ON OTS G9) Boe rf 


Two levels of generalization. There are two well-recognized 
levels of generalization attained in educational research: (1) 
empirical generalizations, and (2) complete scientific explana- 
tion. When one observes some uniformity of nature which he 
holds to be true and for which he has no adequate explanation, 
his knowledge is said to be empirical. If one knew, for ex- 
ample, nothing of the theory of gravitation and planetary 
motion, but had observed the sun to rise in the east each 
morning, his knowledge of the movement of the sun would be 


4W. C. Trow, Scientific Method in Education, pp. 72-93. Boston: Houghton Mifflin 
Co., 1925. 
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empirical. This is only one of many illustrations of this sort 
of knowledge which might be cited. There are, of course, nu- 
merous instances of this type of generalization in specific fields 
such as science and education, thus, empirical education, em- 
pirical agriculture, empirical medicine, etc. Most of the prac- 
tices in the field of education to-day rest not upon complete 
scientific explanation but upon observed uniformities arising 
from either ordinary observation or scientific investigation, for 
which there are as yet no satisfactory explanations. The dis- 
tinguishing feature of empirical knowledge, however, is not 
whether it has its origin ‘in ordinary observation or in scien- 
tific investigation, but whether it is explained and understood. 
It arises not only in agriculture, medicine, or education, but 
in the physical and biological sciences and wherever there 
are observed but unexplained uniformities. 

The aim of every science is to extend knowledge beyond the 
facts of observation to other facts that have not been observed, 
by binding them together into one comprehensive system of 
knowledge. To refer again to the illustration introduced above, 
one desires to know not merely that the sun rises regularly 
in the east, but why it rises in the east and why it is likely 
to do so for some time to come. Even though the majority 
of recorded facts must be empirical and thus are awaiting 
explanation, such knowledge is of minor value in comparison 
with explained knowledge, because it does not permit safe 
and extensive inference. What one desires to know is not 
merely that certain phenomena behave in certain observed 
uniform ways, but why they behave as they do. To make 
such explanations, one must refer the observed uniformities 
to some already established principle of science. One under- 
stands a thing only when he grasps its underlying principles. 
There is in the field of education to-day, for example, a rather 
large amount of superficial investigation of the relative merits 
of different instructional procedures, owing to the absence of 
any fundamental theory of teaching and learning to serve the 
purpose of directing the research and explaining observed uni- 
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formities. Such studies can result only in observed uniformi- 
ties and empirical knowledge. The ultimate goal of educational 
research, however, is knowledge which is understood and ex- 
plained. 

Ways in which conclusions may be stated. Excellent exam- 
ples of different ways in which conclusions may be stated, 
classified according to the following outline, are available; that 
is, the illustrations consist of references to investigations which 
may be examined to note the way in which the generalizations 
have been formulated: * 


A. Statement of conclusions 


1. Concise statements 
a. Answers specifically connected with questions asked in 
statement of problem 
b. Conclusions specifically connected to declarative statements 


of problems 
General statements 


. Recognition of limitation due to faulty or incomplete data 


2 
B. Statement of limitations in conclusions 
I 
2, Recognition of danger of generalization 


C. Application of conclusions 
1. Suggestions for application to practice 
2. Suggestion of problems for further investigation 


Concluding statement. The purpose of research is to derive 
verified generalization. Because of the many pitfalls accom- 
panying the processes of investigation, generalizations should 
be made with great care. Four tests should be painstakingly 
applied to all conclusions before accepting them as valid: (1) 
agreement with fact; (2) agreement with the laws of nature 
already held true; (3) simplicity of formulation; and (4) 
verifiability. In addition to these tests, each investigation 
should be checked for numerous fallacies in reasoning. Al- 
though the great mass of recorded facts, collected through 
the process of research, is empirical and is awaiting explana- 

SW. S. Monroe and M. D. Engelhart, The Techniques of Educational Research, pp. 


50-4. University of Illinois Bulletin, Vol. XXV, No. 19. Bureau of Educational Research 
Bulletin, No. 38. Urbana, Ill.: University of Illinois, 1928. 
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tion, the ultimate goal of the scientific method is complete ex- 
planation of the processes of nature. 


PROBLEMS AND EXERCISES 


1. Cite a number of erroneous generalizations in education, other than 
the examples mentioned in this chapter, and indicate the nature of 
the fallacies involved. 

2. Select a recent educational investigation, and analyze the con- 
clusions reached according to the principles discussed in this 
chapter. 
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CHAPTER XIII 


THE PREPARATION OF THE RESEARCH REPORT 


Introduction. This chapter emphasizes the importance of 
effective reporting of educational research and outlines the 
major elements to be considered in such writing. The most im- 
portant items here discussed are: major divisions of the re- 
port; development, evaluation, and organization of ideas; 
English usage; educational terminology; tables and graphs; 
bibliographical and footnote form; and publication of manu- 
scripts. 

Importance of effective writing. Extended comment is un- 
necessary to indicate the need for clear, forceful expression 
in reporting the results of an educational investigation. In 
many instances, worth-while results secured from an experiment 
or study have been rendered comparatively ineffective through 
a careless and inadequate style of writing and organization. At 
least a part of the energy once devoted to promotion of class- 
room experiments and to convincing field workers of the desira- 
bility of replacing subjective opinion with objective data must 
be diverted to improvement in educational writing. Undoubt- 
edly there is a close relation between clear thinking and a well- 
organized, effective style of writing. The foregoing statements 
are made with a definite recognition of the fact that the most 
fundamental element in determining the value of a research 
report is the scientific character of the material presented. 
However, since it is difficult for those who read a report to 
give undivided attention to content if the form is poor, certain 
requirements commonly accepted by careful writers should be 
observed. Subsequent pages of this chapter set forth briefly 
selected elementary principles of effective writing and organiza- 
tion of material. 
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The significance of reporting as a phase of educational re- 
search and the communicatory function of the report are well 
stated by Monroe and Engelhart: * 


The report of an investigation not only serves to record and com- 
municate the procedure and the results but it also fulfills an important 
function in the process of research. In the act of writing, if it is well 
done, the research worker refines his thinking, and the detailed record 
facilitates the critical testing of the work done. Thus an investigator 
should not consider that he has completed his task until a complete 
report has been prepared. If he is interested in communicating his work 
to others, the report must be well written in order to fulfill that pur- 
pose effectively. 

Usually, in preparing a report of a study, a research worker should 
not confine his purposes to “telling” the reader the answer that he has 
obtained to the problem studied. Instead, he should try to guide the 
reader to think about the problem in such a way that at the conclusion 
of his reading a dependable answer will have been attained. Thus a 
complete report of an investigation should guide the reader (1) to de- 
fine the problem clearly so that he will understand just what questions 
are to be answered and in some cases their relation to other questions, 
(2) to understand the data introduced, especially to be aware of their 
limitations, and (3) to test critically the hypothesis which is to be 
accepted as the answer and as a result to qualify or limit it if the data 
indicate that this should be done. In case other hypotheses are likely to 
occur to the reader, the discussion should guide him to an understand- 
ing of the reasons for their rejection. 


The same authors have prepared an excellent list of criteria 
to be used in evaluating educational writing, which is of serv- 
ice as a check-list in analyzing or criticizing either the work 
of other writers or of one’s self.* The organization of the pres- 
ent chapter is determined to a considerable extent by the 
sequence of topics listed in the outline which follows. In the 
discussion subsequent to the outline the present writers are 
also indebted to Monroe and Engelhart. Some of the nineteen 
items in the following list are grouped in the discussion pre- 
sented in this chapter. 


1 W. S. Monroe and M. D. Engelhart, The Techniques of Educational Research, D. 57. 
University of Illinois Bulletin, Vol. XXV, No. 19. Bureau of Educational Research 


Bulletin, No. 38. Urbana, Nl.: University of Illinois, 1928. 
2 Ibid., pp. 58-61. 
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SUMMARY OF CRITERIA FOR REPORTING RESEARCH 


A. General structure of a report 
1, Major divisions , Na 
(a) Are the following divisions reasonably explicit: definition 
of problem, collection of data, treatment of data, discus- 
sion of each question to be answered, and the con- 
clusion? 
2. Introduction 
(a) Is the problem introduced in such a way that a com- 
petent reader will understand and appreciate the purpose 
of the report? 
(b) Has superfluous material been eliminated from the in- 
troduction? 
3. Definition of problem 
(a) Is the reader given a precise statement of the questions 
to be answered? 
(b) In case the problem is related to other problems, are the 
relations made clear to the reader? 
4. Conclusion 
(a) Is the reader given in convenient form an explicit answer 
to each question included in the “definition of the prob- 
lem”? 


B. Development, evaluation, and organization of ideas 
5. Trend of thought 

(a) As the reader “traces” the writer’s thinking, will he be 
led from a cleariy defined problem to a critical and 
scholarly answer by a route that is satisfying to him? 

(b) Is an encyclopedic enumeration of ideas or facts avoided? 

6. Development of ideas 

(a) Has the writer avoided leaving “gaps” in his “trend of 
thought” for the reader to fill in? 

(b) Have the important ideas been “developed” so that the 
average reader will fully comprehend them? 

(c) Has the writer developed his ideas so completely that no 
points have been overlooked which might leave the reader 
with unanswered questions? 

7. Evaluation of ideas 
(a) Have all irrelevant ideas been eliminated? 


(b) Have the ideas been grouped properly with reference to 
their relative importance? 


8. Accuracy of interpretation 
(a) Have the data been accurately interpreted? 
(6) Do the statements agree with generally accepted opinior 
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and “common sense”? If not, is attention called to such 
disagreements? 

Precision of statement 

(a) Are the statements made so that they will convey to the 
reader exactly the meaning intended? 

(b) Are all statements worded so that ambiguity or indefi- 
niteness is avoided? 


C. Details of structure and form 


10. 


II. 


I2. 


13. 


I4. 


Diction 

(a) Have appropriate words and phrases been used at all 
times? 

(b) Have particular words and phrases been used with a con- 
sistent meaning? 

(c) Have words and phrases to which common practice has 
assigned technical meanings been used correctly? 

(d) Has attention been called explicitly to each word or phrase 
used with an unusual or restricted meaning? 

(e) Has the “overworking” of certain words been avoided? 

Clearness 

(a) Is the vocabulary suitable for the intended audience? 

(b) Are the ideas expressed in simple yet definite language? 

Rhetoric, grammar, spelling, and punctuation 

(a) Have the rhetorical rules relative to unity, coherence, 
and emphasis in sentence and paragraph construction 
been properly observed? 

(b) Have rules of grammar been observed? 

(c) Are all words correctly spelled? 

(d) Has consistency in the plan of punctuation been ob- 
served? 

Form of tables and graphs 

(a) Are the captions of tables at the top and those of graphs 
at the bottom? 

(b) Are the captions, box headings, and other labels suffi- 
ciently complete so that a competent reader will be able 
to understand the table or graph without referring to the 
accompanying text? 

Explanation and interpretation of tables and graphs 

(a) Has the enumeration of the facts summarized in a table 
or diagram been minimized in the accompanying text? 

(b) Is the accompanying text sufficiently complete so that it 
is unnecessary for the reader to refer to the table or dia- 
gram in order to follow the trend of thought? 

(c) Are references to tables and graphs sufficiently explicit 
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so that the reader will have no difficulty in locating the 
correct table or graph? 

(d) In interpreting a table or graph, is the introduction of 
irrelevant facts or comments avoided, so that the trend 
of thought is not broken? 

1g. References to sources of information 

(a) Are bibliographical references given for statements or 

facts taken from the works of other persons? 
16. Bibliographical form 

(a) Are all references both in footnotes and in bibliographies 
given in an approved bibliographical form? 

17. Chapter titles, table of contents, preface, title page, order of 
paging, spacing, kind of paper, and so forth 

(a) Have conventional rules with reference to chapter titles, 
table of contents, and so forth been observed? 

18, Footnotes 

(a) Have footnotes been used to give needed explanations 
or other comments which will make more certain a cor- 
rect and complete understanding by the reader? 

(b) Has material which would tend to break the trend of 
thought but which is desirable to include been placed in 
a footnote or in an appendix? 

19. Miscellaneous 

(a) Have conventional rules with reference to abbreviations, 
division of words, spelling out numbers, and so forth 
been complied with? 


Major divisions of the report. The first step in the prepara- 
tion of a report should be the formulation of an outline of its 
major divisions. A brief, expressing in abbreviated form the 
principal statements to be made under each heading and sub- 
heading, is helpful, representing a more advanced stage of 
thinking than in the case of the outline. 

A report of some length (fifty pages or more) may be appro- 
priately divided into chapters or sections which deal with the 
fundamental problems involved, such as statement of the prob- 
lem, related studies, techniques of collecting data and sources 
of data, results, and conclusions. Short reports usually indicate 
such subdivisions by paragraph or center headings rather than 
by a chapter type of organization. 

Paragraph headings are now considered highly desirable in 
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educational reports and publications: textbooks, monographs, 
bulletins, theses, and articles. If the number of topics is large 
and the writer is unable to organize an effective sequence by 
use of a sheet of paper, cards may be employed. In the case of 
a graduate student who was preparing a long chapter of his 
thesis, some sixty-five subdivisions were listed on as many 
cards, which were shuffled until the desired sequence or or- 
ganization resulted. Paragraph headings may be based on the 
outline or brief and should be prepared before rather than 
after the report is written. Of course it is not necessary that 
each paragraph have a heading ; only when a new unit is begun 
should there be a heading to show the transition in thought. 
One overzealous student not only began each paragraph with 
a heading, but for some unknown reason mistakenly assumed 
that each heading should contain an author’s name. In the 
typewritten manuscript the common practice is to underscore 
paragraph headings, which changes to italics or bold-face type 
in print, depending on preference. Effectively organized re- 
ports also use summary and transitional statements and para- 
graphs to aid the reader in following the author’s basal outline. 
One writer who said some years ago that it was a reflection 
on the intelligence of the reader to make use of paragraph 
and sectional headings wrote an important educational book 
without such mechanical aids. Interestingly enough, the 1931 
revision of this book has an exceilent organization in terms of 
appropriate headings, probably the result of a demand by 
first-edition readers for assistance in following a rather tech- 
nical type of educational thinking. The amount of significant 
literature available to-day is so large that it requires every 
legitimate mechanical aid which publisher and author can 
devise to promote rapid reading and economy of time. 

In the introductory part of the report the problem should 
be stated clearly in order that subsequent discussion may be 
understood. In fact, it is a fairly safe plan in the graduate 
thesis to state the problem in the opening paragraph. It is a 
source of irritation and confusion to the reader to cover sev- 
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eral pages of material dealing with the reasons for undertaking 
the study and its significance before ascertaining the nature 
of the problem itself. The length of the introductory section 
depends in part on whether the writer includes under this 
heading a statement of the sources of the data used, tech- 
niques employed, limitations, historical background, related 
studies, etc. In some instances separate sections or chapters 
may be devoted to such items as those enumerated in the 
preceding sentence, especially where the available material is 
extensive. 

Detailed suggestions concerning the definition of the prob- 
lem have been given in Chapter II. 

Formulation of conclusions is discussed in Chapter XII. 
Briefly and clearly stated conclusions, placed at the end of 
each major division of the report and in the final chapter of 
a lengthy manuscript, are of great assistance to the reader, 
as well as the writer, in ascertaining the generalizations to be 
deduced from the study in question. Use of an outline form 
or a numerical sequence facilitates rapid examination of, and 
reference to, such summaries. 

Development, evaluation, and organization of ideas. The 
tendency of many young writers is to leave gaps or breaks in 
their reports, taking for granted familiarity with the concepts 
and procedures employed. In his own thinking and experience 
the immature writer has used certain ideas so frequently that 
he forgets the reader has not had a similar opportunity. The 
carefully prepared outline and brief mentioned earlier in this 
chapter should aid materially in promoting a continuous and 
unified trend of thought. The skilful writer leads his reader 
step by step through a well-organized analysis of the problem, 
method, data, and conclusions. Poor writing may be due to 
inadequate command of the English language, to hazy thinking 
on the problem in question, or to both. 

Analysis, interpretation, generalization, and evaluation as 
applied to data and ideas are discussed at some length in 
Chapters XI, XII, and XIV. 
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The need for accuracy in the preparation of all phases of 
the report has been well expressed as follows:* 


The price of accuracy is quite evidently eternal vigilance. The de- 
gree to which it is feasible or desirable having been determined in the 
light of purposes and the nature of the data with which we deal, one 
must persistently strive for it by defining terms and simplifying pro- 
cedure while gathering the information, by applying the known theo- 
rems concerning accuracy, by adopting an accuracy routine during the 
compiling, tabulating, and interpreting of the data, and finally by pains- 
taking care during the period when the final report is passing through 
the hands of the printer. 

With particular reference to educational workers, we believe the 
ideals for insuring accuracy are not as high as they should be. Very 
likely this is likewise true of workers in other fields. Not only—as we 
have had repeated occasion to know in our editorial work—do figures 
in different tables of the same report often fail to “check up,” but little 
effort appears to have been made to rid the data of such error as even 
an editor may detect. Some of the most competent workers are so 
much interested in devising tests and increasing the volume of ma- 
terial on which their conclusions rest that they have failed to devote 
the attention to accuracy which, in our judgment, it demands. 


English usage. The writer should have reference to a good 
handbook which deals with the accepted principles of rhetoric 
and grammar. Too frequently faulty sentence structure, il- 
logical paragraphing, poor punctuation, incorrect spelling, and 
other violations of English usage render an otherwise prom- 
ising manuscript virtually worthless. It is unfortunate that 
too often the reader’s attention is distracted from significant 
data and conclusions because of such lapses. It may be sug- 
gested, parenthetically, that simplified spelling systems have 
not yet been generally adopted in scientific writing. Certain 
of the available style manuals for authors are especially valu- 
able.t Other manuals are listed in the chapter bibliography, 

3 “Accuracy,” Journal of Educational Research, VIII (June, 1923), 63-7- 

4George Carver, Points of Style. New York: Thomas Nelson and Sons, 1928. Pp. 86. 

W. R. Good, How to Prepare a Term Report, Ann Arbor, Mich.: Alumni Press, 1931. 
Pp, iv + 28. 

Garland Greever and Easley S. Jones, The Century Handbook of Writing. (Third 
edition.) New York: The Century Co., 1933. Pp. xvi + 272. 


J. H. Hanford and Others, The Nelson Handbook of English. New York: Thomas Nel- 
son and Sons, 1932. Pp. 344- 
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and in addition, certain portions of volumes on research pro- 
cedure, found at the end of this chapter, may be examined, 
for example, Abelson, Almack, Bixler, Campbell, Crawford, 
Good, Odum and Jocher, Reeder, Schluter, Spahr and Swenson, 
and Whitney. 

Educational vocabulary and terminology. An analysis of the 
style employed in educational periodicals ends with the recom- 
mendation that educational writing be done in a straightfor- 
ward, business-like manner suited to technical publications." 
Many educational journals need improvement in this respect. 

Obviously, school workers and writers in education should 
seek to employ a vocabulary as exact and uniform as that used 
by practitioners in medicine or law. However, there is evidence 
of a diversified character to show that such is not the case. 
It seems appropriate to cite certain of the evidence concerning 
inexact and confusing usage of terminology in secondary and 
higher education, as well as to indicate the vocabulary prob- 
lems which confront the reader of textbooks and professional 
treatises in high school and college. Some of the attempts 
toward standardization of our educational nomenclature will 
be enumerated as sources of additional information. Else- 
where, one of the writers has discussed these problems at 
greater length than is possible at this time.* 

An excellent statement of the inconsistencies and lack of 


ba Manual of Style. (Ninth edition.) Chicago: University of Chicago Press, 1927. 
Pp. 39%. 

Manual on Research and Reports. Baltimore: Williams and Wilkins Co., 1931. Pp. 108. 

C. O. S. Mawson, Style Book for Writers and Editors. New York: Thomas Y. Crowell 
Co., 1930. Pp. 213. 

R. Morel Schmitz, Preparing the Research Paper. New York: Richard R, Smith, Inc., 
1931, Pp. 94. 

W. K. Smart, Handbook of Effective Writing. New York: Harper and Brothers, 1931. 
Pp. xx + 292. 

Style Manual of the Government Printing Office. Washington: Government Printing 
Office, 1933. 

S. F. Trelease and E. S. Yule, Preparation of Scientific and Technical Papers. 
(Revised edition.) Baltimore: Williams and Wilkins Co., 1930. 

E. C. Woolley, College Handbook of Composition. Boston: D. C. Heath and Co., 
7928. Pp. xix + 396. 

5 Arthur Minton, “Style in Education Periodicals,” English Journal, College Edition, 
XXIV (November, 1935), 724-7. 

6 Carter V. Good, How to Do Research in Education, Chapter III. Baltimore: Warwick 
and York, 1929. 
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exactness found in the terminology of periodical literature is as 
follows: 7 


One who reads carefully the articles that appear in the successive 
issues of the School Review cannot fail to be struck by the loose use 
of terms which constantly occurs in the educational writings of the 
present day. For example, one author in the March number uses the 
term grades in the title of his article, while another employs the terms 
mark and grade interchangeably. Grade is the term in general use to 
distinguish the successive groups of pupils in their progress through 
the school and should be used exclusively for this purpose. The term 
mark is accurately descriptive and has but one use which is clearly 
understood, There is a similar, but less confusing, use of the terms 
pupil and student in articles in the secondary field. The more careful 
writers use pupil when speaking of the high school and student when 
referring to the college. There is also the confused use of schedule and 
program of recitations. The former is distinctive; the latter is properly 
used only in the expression program of studies which is applied to all 
of the curriculum offerings of a school. School men in other countries 
avoid the confusion into which we have fallen by using the terms 
schedule and time-table. Extracurricular is another descriptive word 
which is now inaccurate. When the activities thus designated first ap- 
peared, they were quite outside the formal studies comprising the cur- 
ticulum, With the broader aims now accepted, the curriculum has 
expanded to include all educative experiences, among which the great 
variety of social activities now organized and directed by the modern 
school are recognized as having an important place. The term extra- 
classroom should be used as more accurately descriptive in designating 
these. 

. .. The claim that teaching is a profession would be strengthened 
by a more exact use of language in our professional journals. 


A number of specific factual studies of the range of ter- 
minology employed in education are available. Even in the 
much discussed field of objectives in secondary education an 
analysis of forty manuals and programs of studies reveals 
106 ways of stating the aims of high schools.* If secondary- 
school workers themselves do not express with some uni- 
formity the goals toward which they are working, pupils and 
parents may well be hazy concerning such objectives. 


7 Franklin W. Johnson, “Educational Terminology,” School Review, XXXII (June, 
1924), 407-8. 

8 Carter V. Good, “The Objectives of Secondary Schools in 1926-27,” Education. 
XLVII (June, 1927), 585-92. 
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A study of the titles of parallel curricula listed in sixty- 
nine city programs of studies and state manuals showed 130 
in use,” while a later investigation of the senior high schools of 
sixty-five school systems revealed 184 curriculum titles.1° The 
courses of study in the social-science departments of twenty- 
nine high schools contained forty-eight different course titles.1! 
Without attempting at this point to suggest the range of cur- 
ricula or courses desirable in secondary schools or to select 
the titles appropriate, it seems clear that any increased uni- 
formity of terminology should facilitate pupil transfer from 
one high school to another and college entrance, as well as 
clarify the thinking and writing of workers in this field. 

Problems of terminology in higher education are of concert: 
to the public-school worker. Most administrators, supervisors, 
and classroom teachers come from the higher institution to 
begin their work in the field, and many return during summer 
sessions for further training. The high school in turn must 
provide guidance for its graduates who enter the college and 
university in large numbers each year. The problem of ar- 
ticulation will prove much less difficult when a common basis 
of understanding has been effected through the employment 
of uniform titles for courses of similar content, use of the 
same terms in evaluation of credit, elimination of duplication 
in course content, etc. The partial neglect of these issues is 
well illustrated by the diversity of the terminology in use, 
the lack of standardization, and the frequent neglect of the 
vocabulary problems of the young student. 

Illustrations may be drawn from several fields. First, data 
are given relative to the size of the technical vocabulary to 
which the young college student is introduced. An investigation 
including ninety-eight students who were asked to prepare a 
statement of what they considered the characteristics of good 


9 Carter V. Good and Raymond E. Good, “Titles of Curriculum Offered or Suggested 
in Secondary Schools,” School Review, XXXV (September, 1927), 503-9. 
10 Carter V. Good and Edward D. Roberts, ‘Curriculum Titles and Curriculum 
Constants in Senior High Schools,” School Review, XXXVI (November, 1928), 679-84. 
11 Carter V. Good, “The Variables of the Senior High School Curriculum and the 
College-Entrance Problem,” School Review, XXXV (November, 1927), 686-91 
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and poor textbooks shows that “too much technical vocabu- 
lary” heads the list of specific sources of difficulty in the text- 
books mentioned? Analysis of ten textbooks in educational 
psychology revealed 559 selected technical terms considered 
important by four psychology instructors. The catalogues of 
155 teacher-training institutions contained a total of 897 
courses in psychology under 182 different course titles.’ 

A college text in zodlogy commonly used in beginning 
courses was found to contain 4,226 technical terms not in- 
cluded in the ten thousand words of the Thorndike list of most 
commonly used words.’ Since teachers of foreign languages 
consider the acquisition of one thousand new words a fair 
achievement for the first year’s work in their field, the tech- 
nical vocabulary in the zodlogy text appears to be about four 
times as great as that which students are expected to master 
in ə foreign language. Such facts indicate that the problem of 
{echnical vocabulary is one worthy of consideration in the 
textbooks of fields other than education. 

At present several available attempts toward stanuardization 
of portions of the vocabulary in education and psychology may 
be mentioned: high-school *® and college ** terminology, finan- 
cial reports of higher institutions and terms and procedures used 
in school statistical reports,” measurement and research, ° 


12S. L. Pressey, L. C. Pressey, and Others, Research Adventures in University 
Teaching, pp. 61-3, 127-33. Bloomington, Ill.: Public School Publishing Co., 1927. 

13 Marion E. MacDonald, “A Catalog Study of Courses in Psychology in State Normal 
Schools and ‘Teachers Colleges,” Educational Administration and Supervision, XIII 
(April, 1927), 272-82. 

14S. L. Pressey, L. C. Pressey, and Others, op. cit., pp. 127-53- 

15 Charles Hughes Johnston, Jesse H. Newlon, and Frank G. Pickell, Junior-Senior 
High School Administration, pp. 65-88. New York: Charles Scribner’s Sons, 1922. 

16 David A, Robertson, Standard Terminology in Education. Educational Record Sup- 
plement (January, 1927). Washington, D. C.: American Council on Education. Pp. 32. 

17 National Committee on Standard Reports for Institutions of Higher Education, 
Suggested Forms jor Financial Reports of Colleges and Universities. Emory University, 
Georgia: The Committee, G. H. New, Secretary. 

H. A. Dawson and E. M. Foster, A Preliminary Study oj Terms and Procedures Used 
in School Statistical Reports. Office of Education Circular, No. 137, February, 1935- 
Washington: Office of Education, 1935- Pp. 16. 

18 Walter S. Monroe, Definitions of the Terminology of Educational Measurements. 
University of Ilinois Bulletin, Vol. XX, No. 6. Bureau of Educational Research Circular, 
No. 13. Urbana, Il.: University of Illinois, 1922. Pp. 18. 

Charles W. Odell A Glossary of Three Hundred Terms Used in Educational Measure- 
ment and Research. University of Illinois Bulletin, Vol. XXV, No. 28. Bureau of Educa; 
tional Research Bulletin, No. 40. Urbana, Il.: University of Illinois, 1928. Pp. 68- 
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psychological terms,* and ambiguous terms in educational 
psychology.” Haggerty is responsible for the compilation 
of a list of 2,584 topics in psychology.” Voegelein has 
prepared a valuable alphabetical classification of activities and 
functions which covers the entire field of education and aims 
to assist educators and librarians in the preparation of refer- 
ences to educational literature and in filing materials.** 

Rugg attempted to find what educational terminology a be- 
ginning teacher should know by analyzing five general 
introductory textbooks in education and by sampling eleven 
periodicals. The study resulted in 2109 different concepts, in- 
cluding derivatives.” Gilchrist sought to determine the educa- 
tional terms which secondary-school teachers must know in 
order to understand present-day educational literature, by an- 
alyzing the articles dealing with secondary education in the 
School Review during the years 1913, 1920, 1926, and 1927." 
Although such studies possess obvious limitations in terms of 
completeness, they offer valuable suggestions to writers, stu- 
dents, teachers of education, and workers in the field. 

Of course, space is not available to present a glossary of 
educational terms, which seems unnecessary in view of the 
several dictionaries devoted to this purpose. 

At this writing certain terminology projects are under way 
or under consideration. A committee of the American Council 
on Education has developed plans for a comprehensive dic- 
tionary of educational terms. A. K. Kurtz and H. A. Edgerton 
have worked out tentatively a dictionary of some 2,000 sta- 


19 Horace B. English, A Student’s Dictionary o, ychologi { j 
ETT eae O ae: y of Psychological Terms, Yellow Springs, 

H. C. Warren, Dictionary of Psychology. Boston: Houghton Mifin Co., 1934. 

20S. M. Corey, “Certain Ambiguous Terms in Educational Psychology,” Journal of 
Educational Psychology, XXII (February, 1931), 131-8. 3 

21 M. E. Haggerty, Topics in Psychology. Minneapolis, Minn.: University of Minnesota 
Press, 1929. 

22L. Belle Voegelein, List of Educational Subject Headings. Columbus, Ohio: Ohio 
State University. 1928. Pp. xiv + 338. 

23 E. U. Rugg, “What Educational Terminology Should a Beginnin; t. 
Educational Administration and Supervision, XVI (March, Seay a ee 

24 Robert S. Gilchrist, “Leading Terms in Educational Literature,” School Review, 
XXXVIII (October, 1930), 622-6. : 
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tistical terms. H. L. Nearpass is preparing a dictionary and 
handbook of educational measurements in which more than 
2,000 terms are defined and discussed. A committee of the 
American Library Association is now formulating definitions 
of library terms. There is in progress a classified list of educa- 
tional subject headings, with an alphabetical index, which 
will include definitions of many terms. 

Tables and graphs. The problems of analysis, classification, 
and interpretation of data are treated at length in Chapter 
XI. Therefore, tabular and graphic presentation will be dis- 
cussed briefly at this time only in terms of form. 

In the interest of brevity, a series of statements may be 
made with respect to construction of tables. 


1. Each series of items which involves frequencies should be pre- 
sented in the form of a table with an appropriate title and number. 
When a table lacks either title or number, both author and reader may 
have difficulty in referring to or locating readily the desired data. 

2. The table heading should not be divided into separate clauses 
or sentences, but should express adequately in a continuous title the 
nature of the data included. To begin the title with such words as 
“showing,” “table showing,” and “distribution of” (usually) is super- 
(uous. 

3. The word TABLE (written in double caps as in this sentence), 
together with the number of the table in Roman numerals, should be 
centered above the table, followed on the next line by the title in 
double caps. 

4. A double ruling should appear below the title of the table. 

s. A single ruling should appear at the end of the table. 

6. Vertical rulings should be inserted as needed, and care should be 
taken that the heading above each particular section applies only to 
that section. Rulings are not necessary on the right and left margins of 
the table. 

7. Uncommon abbreviations should be explained in footnotes be: 
low the table. In general, abbreviations are to be avoided, if possible 

8. Footnotes in connection with the table should be referred to by 
stars, asterisks, or letters rather than by figures. Otherwise, confusion 
with the data of the table might result. 

9. If possible, confine the table to a single page. Unwieldy tables 
in many instances may be broken up into separate tables in order to 
prevent use of a large sheet which has to be folded. Folded tables and 
graphs involve difficulties in binding the report, are easily lost or torn, 
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and wear through where creased. When a table must be continued on a 
second page, the box headings should be repeated at the top of each 
column of data, so as to facilitate ready use of the material; the table 
number, but not the table title, should be repeated, thus, TABLE I 
(Continued). It is sometimes necessary to turn unusually wide tables 
sidewise on the page. 

to. Titles of tables in the list of tables (in the front of the report) 
should be written in single caps. Lists of tables and figures are useful 

` in locating desired data. 

1r. The title of each table in the list of tables should correspond 
exactly with the title on the page where the table appears. 

12. Tables shouid be placed in the manuscript as near the point of 
first reference as possible and should be centered in the page for the 
balanced appearance which results. 

13. Too many groups or columns of data should not be shown in 
one table. It is better to select for each table the data needed to estab- 
lish clearly a limited number of points, in some instances only one or 
two, than to invite confusion as the result of too complicated a table. 


For illustrations of tabular form the reader may examine 
the various tables of this book, as well as the lists of tables 
found among the preliminary pages of many volumes, which will 
aid in locating the tables, although the reader must remember 
that the typewriter lacks the flexibility of printing type and 
cannot reproduce exactly tables found in printed volumes. 

A committee has provided valuable suggestions for the 
preparation of graphs, diagrams, figures, charts and maps.”* 


1. The general arrangement of a diagram should proceed from left 
to right. 

2. Where possible, represent quantities by linear magnitude, as 
areas or volumes are more likely to be misinterpreted, 

3. For a curve, the vertical scale, whenever practicable, should be 
so selected that the zero line will appear in the diagram. 

4. If the zero line of the vertical scale will not normally appear in 
the curve diagram, the zero line should be shown by the use of a hori- 
zontal break in the diagram. 

5. The zero lines of the scales for a curve should be sharply dis- 
tinguished from the other codrdinate lines. 

6. For curves having a scale representing percentages, it is usually 
desirable to emphasize in some distinctive way the 100 per cent line 
used as a basis of comparison. 


25 Report of the Joint Committee on Standards for Graphic Presentation. New York: 
American Society of Mechanical Engineers, 1918. Pp. 50. 


PREPARING THE RESEARCH REPORT 663 


>. When the scale of the diagram refers to dates, and the period 
represented is not a complete unit, it is better not to emphasize the 
first and last ordinates, since such a diagram does not represent the 
beginning and end of time. 

8. When curves are drawn on logarithmic codrdinates, the limiting 
lines of the diagram should each be of some power of ro on the 
logarithmic scale. 

9. It is advisable not to show any more coordinate lines than 
necessary to guide the eye in reading the diagram. 

to. The curve lines of a diagram should be sharply distinguished 
from the ruling. 

tr. In curves representing a series of observations, it is advisable 
whenever possible, to indicate clearly on the diagram all the points rep- 
resenting the separate observations. 

12. The horizontal scale for curves should usually read from left 
to right and the vertical scale from bottom to top. 

13. Figures for the scale of a diagram should be placed at the left 
and at the bottom or along the respective axes. 

14. It is often desirable to include in the diagram the numerical data 
or formulae represented. 

15. If numerical data are not included in the diagram, it is desirable 
to give the data in tabular form accompanying the diagram. 

16. All lettering and all figures in a diagram should be placed so 
as to be easily read from the base as the bottom, or from the right- 
hand edge of the diagram as the bottom. 

17. The title of a diagram should be made as clear and complete 
as possible. Subtitles or descriptions should be added if necessary to 
insure clearness. 


Rugg has provided excellent illustrations of these seventeen 
principles of graphic presentation enumerated above, as well 
as a helpful treatment of tabular, graphic, and statistical 
analysis in general.*° 

Additional suggestions concerning the form of graphs are as 
follows: 


1. Figures should be numbered in Arabic numerals, followed by a 
period and a dash, with the legend placed beneath the figure (not in 
double caps). 

2. Sufficient explanation of symbols should be given below the 
figure. This applies to different cross rulings in a bar diagram, different 
kinds of lines or colors in a line diagram, etc. 


26 Harold Rugg, A Primer of Graphics and Statistics for Teachers, pp. 123-32. Boston: 
Houghton Mifflin Co., 1925. 
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3. Titles of figures in the list of figures (in the front of the re- 
port) should be written in single caps. 3 

4. The title of each figure in the list of figures should correspond 
exactly with the title on the page where the figure appears. 


For additional information concerning tabular and graphic 
presentation of data, reference may be made to almost any 
one of the textbooks of the past few years dealing with statis- 
tical method. Preparation of tables and graphs has received de- 
tailed attention in a number of useful books.” In view of the 
availability of such aids and the variety of graphic procedures, 
it does not seem feasible to reproduce illustrations in this book. 

Bibliographical form. A consistent and complete biblio- 
graphical form should be adopted and used conscientiously. 
If a given form is prescribed by the graduate school, publish- 
ing house, or journal for which the individual is working, of 
course that plan should be followed. Once a given style has 
been adopted, no liberties should be taken with it, even 
though the student thinks he knows of better forms. One of 
the best systems of bibliographical and footnote form is that 
used by the University of Chicago Press in issuing its educa- 
tional publications—the School Review, Elementary School 
Journal, and Supplementary Educational Monographs (Uni- 
versity of Chicago). The Journal of Educational Research 
uses essentially the same style. The illustrations given in this 
chapter follow the Chicago system. 

The form for books should include information concerning 
author, title (underscored on the typewriter and in italics in 
print), place of publication, publisher, date of publication, 
and number of pages, for example, 

Houzrncer, Karl J. Statistical Methods for Students in Education. 
Boston: Ginn and Co., 1928. Pp. viii + 372. 

27 Fay Campbell, Charts. Chicago: Wheeler Publishing Co., 1928. Pp. 96. 

K. G. Karsten, Charts and Graphs. New York: Prentice-Hall, 1923. Pp. 724. 

Bruce D. Mudgett, Statistical Tables and Graphs. Boston: Houghton Mifflin Co.. 
£930. Pp. viii + 194. 

Helen M. Walkér and Walter Durost, Statistical Tables: Their Structure and Use. 
New York: Teachers College, Columbia University, 1935. Pp. 96. 
J. H. Williams, Graphic Methods in Education, Boston: Houghton Mifin Co., 1924. 


Pp. xviii + 320. 
See also footnote 6 in Chapter XI. 
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Monographs, bulletins, and year-books should be listed in 
the same way as books, with the addition of the name of the 
series and the volume (where used) and number of the series, 
for example, 

Lyman, R. L. Summary of Investigations Relating to Grammar, Lan- 
guage, and Composition. Supplementary Educational Monographs, 
No. 36. Chicago: University of Chicago, 1920. Pp. viii + 302. 

Monroe, W. S., and Enceruart, M. D. The Techniques of Educa- 
tional Research. University of Illinois Bulletin, Vol. XXV, No. 19. 
Bureau of Educational Research Bulletin, No. 38. Urbana, Ill.: 
University of Illinois, 1928. Pp. 84. 

McHate, Kathryn, and Others. Changes and Experiments in Liberal- 
Arts Education. Thirty-first Yearbook of the National Society for 
the Study of Education, Part II. Bloomington, Ill.: Public 
School Publishing Co., 1932. Pp. x + 310. 


Periodical references should include author, title (in quota- 
tion marks), exact name of periodical (underscored on the 
typewriter and in italics in print), volume number in Roman 
numerals, month and year, and number of pages inclusive, 
thus, 

Srouremyer, J. H. “The Psychology of the Table of Contents,” Jour- 
nal of Educational Research, XXIV (October, 1931), 227-30. 

Periodical literature in education includes all published 
materials appearing at regularly stated intervals, ranging from 
weeklies such as School and Society to quarterlies like the 
North Central Association Quarterly. As a rule the makeup, 
cover, size, range of topics covered, etc., distinguish educa- 
tional periodicals or journals rather sharply from year-books, 
annual reports, volumes of proceedings, monographs, and occa- 
sional bulletins belonging to a series. Some students become 
confused when attempting to distinguish between an educa- 
tional journal or periodical such as the Educational Research 
Bulletin (Ohio State University) which appears monthly dur- 
ing the school year, and occasional monographs or so-called 
bulletins, which may belong to a series, but usually deal with 
a single major problem and in form approximate the makeup 
for books. 
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Unpublished material, such as a graduate thesis or term 
paper, may be listed as follows : 


GRINNELL, John Erle. “The Rise of the North Central Association of 
Colleges and Secondary Schools.” Unpublished Doctor's thesis. 
Stanford, Calif.: Stanford University, 1934. Pp. 400. 


Legal material is listed as follows: 


Atlantic Reporter, 1885-1927. 

Legislative Acts of Ohio, 1850-1900. 

Reports of the Supreme Court of Ohio, 1920-27. 
Session Laws of Wisconsin, 1920-26. 


The bibliography should be arranged alphabetically by sur- 
names of authors. Underscoring in a typewritten manuscript 
is italicized in print. The bibliography ordinarily is placed at 
the end of the report rather than at the ends of the various 
sections or chapters, except in the case of textbooks. If it is 


rather long, there is some advantage in grouping the refer- 


ences under appropriate subject or topical headings (but not 
in terms of the form in which printed), maintaining the alpha- 
betical arrangement under each separate heading. Annotations 
render the bibliography more valuable to the reader. A group 
of references on creating social intelligence, with unusually 
full annotations, has been published as a “descriptive biblio- 
graphy.” °° 

Certain suggestions or cautions concerning bibliographical 
form may be summarized briefly : 


1. Be consistent in the use of the form adopted. 

2. Arrange the bibliography alphabetically by surnames of authors. 

3. Avoid abbreviations of journals, organizations, institutions, etc. 

4. Academic and professional titles such as Doctor and Professor 
are unnecessary in scientific writing. 

5. Use the home office of the publishing house represented. 


Two helpful sources present numerous examples of biblio- 


28 ‘Creating Social Intelligence: A Descriptive Bibliography,” Research Bulletin ol 
24e National Education Association, XIII (May, 1935), 71-159. 
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graphical form, the first including a rather detailed card for 
accumulating references.?° 

The extent to which bibliographies have been used is shown 
in an analysis of 144 periodicals in psychology and related 
fields over the period 1900-1927; 2,134 bibliographies were 
discovered.*° Writers have stressed the importance of a canvass 
of related literature by speaking of the bibliographical survey 
as the “pilot” of research.” 

Footnote form. The form of a footnote reference is the same 
as for a bibliographical reference except that the author's 
name is written in the normal order, as John S. Smith, followed 
by a comma instead of a period. When references to the same 
publication occur on the same page, Ibid., followed by the 
page reference may be used, for example, Ibid., p. 28. When 
a reference has been given in full in a footnote less than four 
or five pages back, the author’s name followed by of. cit. and 
page reference may be used, for example, John S. Smith, op. 
cit., p. 72. Other possible forms are listed in the style manuals, 
cited in this chapter, but the two given here will take care of 
the reader’s needs. When there is a large number of citations 
in a report, it is recommended that the author or investiga- 
tion to which reference is made be followed by numerals in 
parentheses, indicating the proper item in the bibliography, 
instead of giving the reference in a footnote. Thus, Smith’s 
study (21:105) means that bibliographical item No. 21 con- 
tains on p. 105 the data cited. Footnotes should be separated 
by a line from the rest of the page in a typed manuscript, 
and numbered in Arabics consecutively throughout a given 
chapter. 

Interesting illustrations of erroneous citations are given in 


29 Carter Alexander, How to Locate Educational Information and Data, pp. 52-64. 
New York: Bureau of Publications, Teachers College, Columbia University, 1935. 

Eleanor M. Witmer and Ethel M. Feagley, 4 Beginners Guide to Bibliography with 
Examples Drawn from the Field of Education. (Revised edition.) New York: Teachers 
College Library, Columbia University, 1935. Pp. 32. 

30C. M. Louttit, “The Use of Bibliographies in Psychology,” Psychological Review, 
XXXVI (July, 1929), 341-7- 

31 W. A. Hamor and L. W. Bass, “Bibliochresis: The Pilot of Research,” Science, 
LXXI (April 11, 1930), 375-8. 
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an article dealing with scientific publications.** The many 
obvious mistakes made in footnote and bibliographical refer- 
ences indicate the need for especial care in copying citations 
to the sources used. 

Whenever material has been quoted directly or indirectly, 
reference by means of the raised Arabic numeral in the body 
of the page should be made to the appropriate footnote, or 
the plan of cross-reference to the bibliography, described 
above, should be used. The practice of giving such recognition 
for material, ideas, and points of view secured from others is 
only intellectual honesty. The individual who violates this prin- 
ciple is guilty of plagiarism or intellectual theft. Available 
discussions refer to striking illustrations of flagrant “literary 
piracy.” * 

There is at least one instance in which a student was 
awarded a Doctor’s degree from one of the best known uni- 
versities on the basis of a dissertation, the most important 
chapter of which was largely borrowed from a new book in 
the same field, with no word of recognition. Apparently the 
candidate was provincially ignorant or intellectually dishonest, 
and his reading committee lax in evaluating the work in terms 
of up-to-date publications on the same subject. Not until 
after the degree was awarded did the officers of this university 
discover the theft through information presented by workers 
from another institution. 

Failure to give full credit in the field of research has been 
referred to as Pythagoreanism. Pythagoras was a Greek scien- 
tist of the sixth century, B.c., credited with contributions to 
mathematics, astronomy, and physical science.** He is said to 

32 G. A. Shull, “Erroneous Citations and Titles of Scientific Papers,” Science, LXXIII 
(April 3, 1931), 363-4. Al ta ote > x 
Re 2: E (oes or Responsibility in Scientific Publication,” Science, 

C. W. Stiles, “Absent-Mindedness as a Factor in Professional Ethics,” Science, LXXI 
(January 24, 1930), 100-1. 

H. B. Van Hoesen, “Literary Piracy,” School and Society, XXXII (November 15, 
1930), 672-3. 

J. E. W. Wallin, “Literary Piracy,” School and Society, XXXII (October 18, 1930), 


527-9. 
34T. A. Boyd, Research, pp. 277-82. New York: D. Appleton-Century Co., 1935. 


— 
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have founded a secret society, the members of which were 
pledged to reveal all their ideas and discoveries to him alone. 
Unfortunately some workers to-day take credit for discoveries 
which they did not make, just as in the case of Pythagoras. 
Nothing could be more detrimental to the morale of an insti- 
tution, school system, research department, or laboratory than 
to have some ranking officer take full credit for the work 
done by his subordinates. This means that ordinarily research 
should be reported or published under the name of the person 
who conducted the investigation. 

Short quotations of a line or two may be placed in quotation 
marks within the paragraph, while longer quotations should 
be single spaced, without quotation marks, in a separate para- 
graph. If copyrighted material is to be quoted in a published 
report, permission to do so should be secured from the pub- 
lisher or author (as a rule, the former); usually such permis- 
sion is graciously given. It is not necessary to secure permission 
to quote material in an unpublished manuscript. The various 
chapters of this book provide numerous examples of footnote 
usage. 

Examples of footnote form in the case of legal materials are 
as follows: 


1 Barry v. McCollom, 81 Conn. 293, 129 Am. St. Rep. 215, 70 Atl. 


1035. 
2 Laws of Nebraska, 1919, Chap. 44, sec. 843. 
3 Burn’s Annotated Statutes of Indiana, 1925, Vol. II, sec. 6448. 
4 School Laws of Pennsylvania, 1925, Sec. 1432. 
5 Compiled Statutes of Nebraska, 1922, sec. 6509. 
6 Revised Statutes of Kansas, annotated, 1923, act. 48, sec. 28. 
7 Smith v. Johnson, 105 Neb. 61, 178 N.W. 835, 12 ALR. 231. 


Preliminary pages and miscellaneous items. Title page, 
preface, table of contents, list of figures, list of tables, and 
index hardly require comment as mechanical aids valuable 
to the reader for orientation purposes and for the ready loca- 
tion of material in the manuscript or printed report. Ordinarily 
the typewritten report does not have an index, and in such 
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cases it may be desirable to make the table of contents more 
elaborate than for an indexed printed report. The type of 
material included in the various preliminary pages of the 
report and the appropriate form are illustrated in the corre- 
sponding pages of this book. 

White bond paper of approximately sixteen-pound weight 
is preferable, and a black ribbon and black carbons should be 
used in the preparation of the typewritten manuscript. Mar- 
gins should be reasonably wide, approximately 1 to 114 inches 
on all four edges of the page. The same margins should be left 
on pages containing charts, graphs, statistical tables, maps, 
etc. The typewritten manuscript should be written.on one 
side only and double spaced, except in the case of quotations, 
which are single spaced. In the preparation of the first draft, 
it is desirable and economical to place tables of some length 
on separate pages, as well as to begin major subdivisions on 
a new page. In this way insertions are possible, and a con- 
siderable amount of retyping is avoided. Scissors and paste 
frequently are valuable aids in assembling the first draft and 
in making revisions. Valuable time should not be wasted in 
unnecessary copying or typing, when cutting and pasting will 
aid in preparing a readable draft, 

Unnecessary division of a word at the end of a line should 
be avoided. If it is necessary to divide a word, the conventional 
rules should be followed, according to the natural divisions in 
correct pronunciation. “Simplified” spelling is undesirable in 
academic and scientific writing. 

Numbers below one hundred and round numbers such as 
five hundred, ten thousand, and one million should be spelled 
out, as well as any number at the beginning of a sentence, Spell 
out “per cent” rather than use the symbol %. Decimals used 
in expressing money and per cent, and enumerations going as 
high as five or six separate items should be expressed in 
figures. 

Scientific writing should he done impersonally. The editorial 
“we” is especially objectionabie. 
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Publication of the manuscript. Valuable studies, reports, 
and theses may be published either in full or in summary 
form in order that the generalizations formulated may be ap- 
plied in the improvement of educational procedure. Of course 
the writer must consider carefully the characteristics of his 
prospective audience and the medium of publication contem- 
plated. If submitted to the wrong publisher or editorial staff, 
the best of manuscripts will be returned. The annual year-book 
of the Educational Press Association of America contains a 
list of the available educational periodicals. The information 
in Chapter III of the present book should aid in selecting a 
prospective avenue of publication. The appendix contains lists 
of journals in subject-matter fields. 

Usually publication of an article in an educational journal 
is considered a professional service and an academic recog- 
nition, without tangible return other than reprints or copies 
of the periodical, furnished without cost in most cases. If 
the suggestions made in this chapter are followed, much edi- 
torial labor and revision by both author and editor will be 
avoided. The uninitiated field worker and student hardly sus- 
pect the way in which editors make readable reports out of _ 
almost hopeless first drafts of manuscripts in which was hid- 
den a gleam of truth. This same statement may be made with 
respect to the work done by many university professors in 
editing graduate theses. It should be emphasized that this type 
of revision belongs properly on. the shoulders of the individual 
who prepares the initial draft of the report. Such training 
should have been secured before the graduate-school period. 

Most journals furnish the author galley proof of his mate- 
rial, while in the case of books he receives both galley and 
page proof. It should be emphasized that changes in proof are 
expensive, especially in page proof. The writer is, therefore, 
obliged to have his copy in as nearly perfect form as possible 
before sending it to press. 

Eells reports an analysis of the editorial policies of forty- 
nine leading journals which should be valuable to those sub- 
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mitting manuscripts for publication.” He asked questions as 
follows: 


1. Do you accept unsolicited contributions for publication from 
graduate students in education? 

2, Is there any restriction to the field you attempt to cover other 
than that indicated by the title of your journal? e.g., elementary, sec- 
ondary, collegiate, research, special topics, etc.? 

3. What is the desirable or maximum length of articles that you 
ordinarily consider favorably for publication? 

4. Do you prefer charts? 

5. Do you prefer tables? 

6. Do you pay for contributions accepted? 

7. If so, at what rate? 

8. Do you furnish the author, without cost, extra copies of an 
issue containing his article? 

g. Are reprints furnished, without cost, to the author? If so, how 
many? 

io. If reprints are not furnished without cost, what is the approxi- 
mate charge per page for four pages, one hundred copies? Three hun- 
dred copies? 

11. Do you send the author proof of articles accepted, before pub- 
lication? 

12. If you have any printed instructions or suggestions for con- 
tributors, please send a copy. 

13. Please state some criteria which you employ to determine 
whether a research article is worthy of publication in your journal. 


The answers given are too detailed and varied to summarize 
here. The reader may refer to Eells’ analysis for the policy of 
any one of the forty-nine journals, 

For the benefit of the writer, whether a beginner or one of 
experience, proofreader’s marks in common use are reproduced 
on the next page.°* 

Concluding statement. The foregoing discussion of the 
preparation of the research report has been directed mainly 
to the field investigator and graduate student, in many in- 
stances one and the same person. It is hoped that such super- 
visors of educational investigation as directors of research 

35 Walter C. Eells, “Publication of Educational Research,” Journal of Educational 


Research, XXXII! (January, 1931), 31-42. 
86 Compiled by the University of Chicago Press. 
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PROOFREADER’S MARKS 
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bureaus, progressive school supervisors, and graduate profes- 
sors are thoroughly acquainted with these elementary prin- 
ciples of good writing. However, it must be confessed that this 
hope is far from realized, as witnessed by the number of poorly 
prepared manuscripts received by all who serve in any editorial 
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capacity. Due appreciation of the work of those who edit 
educational reports, whether books, magazine articles, theses, 
surveys, classroom experiments, courses of study, or other field 
projects should be expressed.*7 


A silent aid if not a “ghost writer” stands behind the finished work 
of many a successful author. He (or she) is the editor who labors for 
long hours over the manuscript, the galleys, and the page proof. In the 
finished production his name rarely appears; yet to him must go the 
credit for many a succinct phrase, for many an appropriate word, for a 
large part of the mechanical correctness, appropriateness, and beauty, 
and for the general form and style of the completed work. With him 
work the other less well-known technicians who have to do with the 
making of books and magazines. To the editor and his helpers the au- 
thor owes, but seldom gives, their due of praise and thanks. 

The making of a book or magazine is no simple task. Eternal vigi- 
lance is the price of perfection. In the securing of fine precision in dic- 
tion, beauty of style, and mechanical beauty no person has quite so 
much influence as the careful editor. He catches the careless author at 
every turn. He has a fine ear attuned to dissonant words and phrases, a 
fine eye for the detection of misplaced marks of punctuation, a posi- 
tive genius for the insertion of the elusive comma, the sturdy semi» 
colon, and the stalwart colon. To him the loose sentence is a simple 
thing, the split infinitive a matter of slight difficulty, and the misplaced 
phrase a problem of simple transposition. The printer’s legend “one 
line short” is a trivial question of mere language ingenuity. With con. 
summate care he puts in a shim here, tightens a bolt there, and adds 
the ever-needed lockwasher to keep the whole thing snug and workman- 
like. Yet in all of this skilled and exacting work, the editor carries on 
in a modest manner; he is courteous and kindly. He constantly says to 
the struggling author, “Would you prefer this, or shall we let the 
original stand?” Truly in the making of books and magazines he has a 
grand and noble part. 

In the making of educational books, the editor has a very large 
share. Such work is exacting and technical, Facts are facts, principles 
are principles, and must be accurately stated. And it may be said, 
without casting reflections upon any one, that writing is an art which 
many educational workers do not possess. The fine appearance, the 
mechanical correctness, and the smooth and effective style of many 
of our educational books and magazines, therefore, owe a great deal 
to the ever present, but generally silent, editor. Praise to him, and to 
her—for in an astonishingly large number of cases it is “her.” Women 
seem to have a special aptitude for the making of magazines and books. 


37 “Praise to the Editor,” University of Pittsburgh School of Education Journal, Tt 
(January-February, 1928), 58-9. 


PREPARING THE RESEARCH REPORT 675 


PROBLEMS AND EXERCISES 


1. Examine one of the bibliographies prepared in connection with 
earlier problem assignments and note whether all of the various pos- 
sible types and forms of references are represented, and follow some 
complete and uniform bibliographical style. Make any needed correc- 
tions. 

2. Select some completed study, and analyze it in terms of desirable 
standards for reporting research, as developed in this chapter. If 
desired, use the check-list of 19 major items found near the beginning 
of this chapter. 
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CHAPTER XIV 


STANDARDS FOR EVALUATION OF WRITING, RESEARCH, 
AND PROCEDURE IN EDUCATION 


The public-school worker and student of education should 
have standards for use: (1) in the evaluation of the large 
volume of literature and research which appears annually, 
(2) for guidance in the preparation of their own reports, and 
(3) for analysis of practice and new movements in education. 
All of the chapters of this book are of some assistance in 
providing criteria for evaluation of the various phases of edu- 
cational thinking, research, and practice. Therefore, the reader 
is referred to the appropriate chapter and bibliography de- 
voted to his given problem. For example, Chapter II on the 
selection of a problem for investigation discusses the extent 
to which various fields and problems have been investigated, 
accomplishments of research in given areas, analyses of trends, 
critiques of investigational effort, educational prophecies, re- 
search under way, needed studies, and criteria for selecting a 
problem. Chapter IIT, on the survey of related literature, in- 
cludes detailed information concerning the availability of 
guides to the various types of educational writing, evaluations 
of published materials, and analytical summaries of research 
in a variety of fields. The nineteen points outlined and dis- 
cussed in Chapter XIII, as significant in the preparation of the 
written report, are of almost equal importance as criteria or 
standards for use in determining the effectiveness of educa- 
tional literature. Criteria for evaluating the various investiga- 
tional methods are found in other appropriate chapters of this 
book. 

In the present chapter no attempt is made to deal directly 


with the rather elusive question of whether education is sci- 
670 


680 EDUCATIONAL RESEARCH 


ence, philosophy, or art, or some appropriate combination 
of each of these. Other chapters (I, IV, XI, and XII) con- 
cerned with the characteristics of scientific method, use of 
hypotheses, interpretation of data, and formulation of con- 
clusions and generalizations provide assistance in discerning 
the relationship between philosophy, theory, logic, art, and 
science, as does the bibliography of the present chapter. Also, 
detailed information concerning the evaluation of given educa- 
tional procedures, plans, practices, or theories must be secured 
from this chapter bibliography, as well as from the references 
in the chapters (II and III) on the selection of a problem and 
the survey of related literature. Therefore, the subsequent dis- 
cussion covers the standards or criteria which may be applied 
in evaluating written reports of educational research or even 
non-investigational published materials, including professional 
books and monographs, periodical literature, theses, school 
textbooks, and courses of study. 


EVALUATION OF EDUCATIONAL LITERATURE AND 
CURRENT RESEARCH 


Evaluation and the consumer of research. An extended 
but excellent statement of the need for evaluation of research 
before attempting to apply its results to educational prac- 
tice follows: * 


For three decades, at an ever increasing tempo, the results of educa- 
tional research have been accumulating. The sheer bulk of research 
literature is now in 1935 little short of astounding. The curriculum 
worker naturally turns to this vast storehouse of material for guidance 
in the solution of his problems—naturally, because he has been led to 
expect help from this quarter. Has not a considerable part of his train- 
ing been specifically devoted to building in him confidence in research 
and in what research will do for education? 

The curriculum worker comes, then, trustingly to educational re- 
search, and what happens? (a) He may be disappointed to discover not 
a single investigation on the point of his interest. (b) Or, he may find 


1W. A. Brownell, “The Field Worker as a Consumer and Producer of- Research,” 
Journal of Educational Research, XXIX (October, 1935), 144-8. 
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one, or perhaps two studies, which he accepts happily and the con- 
clusions of which he uncritically incorporates into his revised cur- 
riculum. (c) Still again, he may, if he reads widely enough, discover a 
number of studies which report inconsistent and even antagonistic find- 
ings. What now? Whose research is he to follow? Would that he had 
not been so ambitious! It would have been so much more satisfying 
to have stopped with the first study he found.* 

In any event the curriculum worker has not solved his problem by 
locating relevant research, His obligation is not the one which is 
usually stressed, namely, that of translating research into school prac- 
tice. He has a more fundamental obligation, that of first determining 
what research, or which research, if any, to accept. Laudable as it is 
in motive, his eagerness to advance the cause of science in education 
by honoring its products is apt to be harmful in its consequences. He 
may be guilty of fostering error, of perpetuating mistakes. All research 
is not equally good. It is not even all good, Some of it is misleading; 
and some of it is bad. The printed word, especially when supported, 
or better, accompanied, by tables, statistical constants, and graphs still 
carries undeserved prestige. Uncritical acceptance of pseudo-research 
creates a demand for, and a supply of, more unscientific effort of the 
same kind. ğ 

The nature of pseudo-scientific research in education has been too 
frequently delineated to justify more than a passing word here. So far 
as the field worker is concerned, perhaps the most important caution 
relates to the interpretation of experimental results. Attention has been 
repeatedly called to the fact that the majority of educational studies 
are fragmentary; they deal with isolated problems. Even when such 
studies are technically accurate and when the findings are derived from 
data by approved statistical procedures, there remains the vital ques- 
tion, What does the study mean? The significance pointed out by the 
investigator himself is not always to be implicitly trusted. On the con- 
trary, the investigator may be too close to his study to see it in propet 
perspective. After all, back of most serious research inquiries lies a 
question and a doubt, and associated with that doubt is some hy- 
pothesis. It is by no means uncommon for that hypothesis to pass over 
into bias which, even when wholly unconscious, may wreck the scien- 
tific worth of the study. 

Yes, the field worker must be a cautious consumer of research. He 
will be the first to welcome valid research findings, but he will satisfy 
himself that the findings are valid. He will read all the research litera- 
ture on his problem. He will recognize that each inquiry represents one 
attack upon the question and that many other attacks may be possible. 


2 One suspects sometimes that textbook writers in education have hit upon this happy 
expedient. This or that educational procedure is recommended on the basis of a single 
citation, others, perhaps better, being omitted—or unknown. 
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He will critically scrutinize the data reported, not hesitating to leave 
the data themselves to weigh their theoretical and practical implica- 
tions,’ He will temper his enthusiasm for research and his faith in its 
ultimate contributions with a sympathetic appreciation of its limita- 
tions. 


Because of the scattered sources in which the consumer of 
research must look for materials, a rather novel proposal has 
been made for improvement of publications in education and 
psyschology : * 


1. Classify research in these fields as to scientific relationships and 
utility. 

2. Assign to magazines of national circulation as much material in 
a given field as they would normally select from the whole field, com- 
bining those left over, or starting new ones if needed. Restrict each 
magazine to its own type of material. 

3. Require all research material to be submitted to a board of read- 
ers whose duty it would be to classify, edit, abstract, and distribute 
such material. 

4. Sign agreements with graduate schools to submit abstracts of 
theses written in article form, or to file copies of theses with a central 
organization to which the board of readers would have access, 

5. Provide translators adequately trained in psychology to make 
available important foreign research without delay. 

6. Offer to specialized and general magazines abstracts of articles 
related to their fields or articles overlapping. 

7. Draw up a set of standards to increase the value of articles 
by adequate presentation. 

8. Collect articles which belong in the field of education or psy- 
chology, but which have wandered for publication into other fields, 
and abstract them completely. 


9. Eventually summarize valid research material of the past in 
all fields, 


At this writing a subcommittee of the American Council on 
Education is considering a proposal for a comprehensive evalu- 
ation, synthesis, and interpretation of the contributions of 
educational research, to be published in one volume. 


3 An excellent example of the need for just such a careful Consideration of research. 
a consideration which carries the inquirer beyond the data of the studies reported, is 
found in Buckingham’s recent critique of the experimental studies on the value ot 
instruction in primary arithmetic. See: B. R, Buckingham, “When to Begin thr 
Teaching of Arithmetic,” Childhood Education, XI (May, 1035), 339-43. 

4V. C. Smith, “A Consumer’s Code for Research,” School and Society, XXXIX 
(March 24, 1934), 371-2, 
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Professional books, monographs, and bulletins. Each year 
in School and Society there is published a relatively complete 
list of American educational books, monographs, and bulletins 
issued during the preceding year. The number for 1935 is 823 
and for 1934, 765, as compared with 813 titles reported in 1933. 
The list is subdivided into twenty-eight fields.° Since field 
workers, college professors, students of education, and li- 
brarians have neither the time nor opportunity to examine 
such a large number of titles for purposes of evaluation with 
a view to professional reading, classroom use, or purchase, 
assistance has been provided in the form of a group of sixty 
selected books of the year,® which are starred in the complete 
School and Society list. This work is done by the Enoch Pratt 
Free Library of Baltimore for the American Library Associa- 
tion and the Journal of the National Education Association. 
Selection of the sixty books of the year is based on an examina- 
tion of the publications themselves, analysis of several 
hundred book reviews, and the scoring and comments of 
educational specialists in the various fields represented. It is 
recognized that such an attempt to select so small a percentage 
of the total titles available means the omission of many worth- 
while volumes, particularly in a subdivision of education where 
unusual productivity has taken place. It is not improbable 
that in some instances the specialists in the field pass judg- 
ment on the merits of new books without having examined 
them carefully, possibly not at all. In spite of such limitations, 
this well-balanced group of sixty books of the year proves 
annually of increasing value in the guidance of teachers, stu- 
dents, and librarians. 

Other book lists, such as those published by the Research 
Division of the N. E. A., the American Library Association,’ 

5 “Educational Books of 1935,” School and Society, XLIII (March 28, 1936), 409-26. 


“Educational Books of 1934,” School and Society, XLI (March 30, 1035), 432-48. 
6 “Sixty Educational Books of 1935,” Journal of the National Education Association, 


XXV (April, 1936), 121-4. 5 > aed 
“Sixty Educational Books of 1934,” Journal of the National Education Association, 


XXIV (April, 1935), 127-30. k È 
7 American Library Association, Books and Pamphlets on Library Work, Chicago: 


American Library Association. Annually. 
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and in authoritative professional books, are very helpful in 
determining the merits of educational publications. Of course 
it is not to be assumed that all of the books appearing in such 
lists are of uniformly high quality. The Education Index 
(under the general heading of “book lists” with appropriate 
subheadings), the Readers’ Guide (for general lists, under the 
heading of “books and reading” with subheadings), the United 
States Catalogue (under the heading of “books and reading”), 
the International Index (under the heading of “books and read- 
ing”), and a number of indexes covering special fields are 
helpful in locating useful book lists. Alexander has prepared 
a partial roster of the book lists cited by the Education Index 
up to December, 1934, as a part of a valuable discussion of the 
location and use of book lists in education.® 

Book reviews. For assistance in keeping in touch with cur- 
rent literature and in its evaluation, educational workers, to a 
large extent, must rely on reviews published in periodicals.° 
The professional journals in the field of education may be 
classified roughly into three groups with respect to the type 
of book review published. A limited number of these journals 
provide analytical, critical estimates of new books, prepared 
by specialists in the fields represented. Certain other periodi- 
cals ‘conduct rather uncritical, informal, occasionally ‘“gush- 
ing,’ and almost always complimentary “book chats” 
concerning current literature in the field. A third group of 
journals publishes little more than brief book notices which 
in many instances have been culled from the preface of the 
new book or from the publisher’s advertisement. Undoubtedly 
the critical point of view, thoroughness, and professional tone 
embodied in the book reviews of the first class of journals men- 
tioned above appeal to the serious educational worker. Among 
the more general educational journals which give especial at- 
tention to their book-reviews departments are: Educational 


Carter Alexander, How to Locate Educational Information and Data, pp. 171-2 
183-6. New York: Bureau of Publications, Teachers College, Columbia University, 1935. 

9 Subsequent paragraphs on book reviews are adapted from Carter V. Good, “The 
Art of Book Reviewing,” School and Society, XXIV (July 31, 1926), 142-3. 


ns 
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Administration and Supervision, Elementary School Journal, 
Journal of Educational Psychology, Journal of Educationat 
Research, Journal of Higher Education, Junior College Jour- 
nal, and School Review. 

The Book Review Digest provides indexes of important 
books and abstracts of reviews published in leading English 
and American periodicals, although this digest is by no means 
complete for the field of education. It appears monthly, cumu- 
lating semi-annually, annually, and at intervals of several 
years. The Education Index, under the heading of “book re- 
views,” is helpful in locating such evaluations. Reviews of 
encyclopedias and sets of books are available in the Subscrip- 
tion Books Bulletin, which appears quarterly with an annual 
index.° The most likely place in which to locate reviews of 
elementary- and secondary-school textbooks is the journal de- 
voted to the level or school subject in question, six months to a 
year after publication of the book in question. An extended 
list of periodicals featuring book reviews of interest to educa- 
tional workers has been classified according to fields or school 
subjects.1? The same source contains a rather extended discus- 
sion of the characteristics of book reviews and the methods 
of locating such evaluations.’* Editorial and news columns of 
educational journals sometimes comment on new books and 
periodical articles. 

A carefully written book review should include three fairly 
distinct parts. An introductory statement or paragraph may 
give the setting of the “stage” upon which the new book 
appears or a brief comment on the need for the new publica- 
tion. Frequently it is the skill with which this introductory 
paragraph is written that attracts the initial attention of the 
reader and lends tone to the remainder of the review. 

In the second place, it is the duty of the reviewer to give a 
brief summary of the contents of the book under considera» 


10 Subscription Books Bulletin. Chicago: American Library Association 
12 Carter Alexander, op. cit., PP. 177-80. 
12 bid.. pp. W717. 
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tion, together with such critical comment as he deems neces- 
sary. Too frequently reviewers lack the skill necessary to 
condense a large amount of material into a small space and 
either extend the summary to unusual length or neglect to 
give a statement of the contents of the volume. In the latter 
instance the reader secures from the review little more than 
the personal impressions and generalizations of the reviewer. 
However, it must be recognized that in some cases the general 
evaluation of a recognized authority is worth a great deal, 
even though he has neglected to present concrete evidence 
from the volume in question as a basis for his expressed 
opinion. Undoubtedly this type of review would take on added 
meaning if adequately documented with factual evidence 
taken from the book under review. 

In the third place, a review should present a critical evalua- 
tion and summary statement of the place the book seems des- 
tined to fill in its given field. This third part may not be defi- 
nitely separated in every instance from the second phase, 
although such a brief statement at the close of the review 
serves to summarize concisely the conclusion reached. 

Discriminating readers of reviews will find that even well- 
written analyses of books vary considerably in style. In some 
instances the reviewer prepares an essay presenting his own 
point of view concerning the field of the book under considera- 
tion, When written by a master, this discussion may be even 
more stimulating than the book itself, although little insight 
into the content of the new volume is provided. Another type 
of review is little more than a summary of major conclusions 
reached, together with a list of major problems treated or of 
chapter headings. The strictly summary review does give a 
table-of-contents overview, but lacks critical analysis and 
evaluation and injection of the personality of the reviewer. 
A third type of review emphasizes critical and analytical treat- 
ment which is the approach of most interest to the careful 
student of education. However, there are possibilities for com- 
bination of all three types of treatment into a well-balanced 
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review. In fact, few reviews follow exclusively a single type 
of approach. 

Tt should be stated that the critical review is not advocated 
universally by students of educational literature. Two con- 
trasting points of view are presented in the quotations which 
follow. Averill and Mueller analyzed book reviews which ap- 
peared during 1926 in seven educational journals in terms of 
the length of the review, practice with regard to signing re- 
views, and the reviewer’s attitude (favorable, unfavorable, 
doubtful, or impersonal). Their recommendation relative te 
printed evaluations of books in education follows: ** 


It would seem to the present writers that the only proper function 
of reviewers should be to present the aims and general scope of the 
books which they review in a wholly impersonal and unbiased way. It 
should reasonably be left to the intelligence of the reader to form his 
own judgment concerning a book’s merits or demerits, and the likeli- 
hood of its being a desirable volume for him to add to his list. As it is, 
too often a reviewer is inclined either, on the one hand, to cater to the 
good will of publishing houses or, on the other, to express freely the 
favor or disfavor with which he personally—and often narrowly—re- 
acts to a new volume according as it is or is not in line with his own 
individual philosophy or viewpoint. 


Monroe and Hull analyzed 548 reviews of one hundred edu- 
cational books published between 1922 and 1925. Their con- 
clusions substantiate the point of view of the present chapter, 
urging a more critical and analytical type of book review.'* 


Book reviews in general may be criticized upon at least two grounds: 
they are inadequate and they are non-critical. Their inadequacy lies in 
their failure to give sufficient information about the book. Too many 
reviews consist of only a few general statements describing the nature 
of the content and make no reference to many specific details that 
should be mentioned. In addition to a general description of the con- 
tent, reviews commonly mention such items as purpose of the book, 
intended audience, and the number of pages. Except for the latter item, 
mechanical features, such as binding, size of type, quality of paper, and 


13 Lawrence A. Averill and Alfred D. Mueller, “Analysis of Book Reviews,” School 


and Society, XXVI (December 31, 1927), 848-9. 
14 W. S. Monroe and Mabel R. Hull, “A Critical Review of Book Reviews,” School and 


Society, XXIX (April 20, 1929), 521-6. 
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illustrative material, are rarely mentioned. Very few reviews compare 
the book reviewed, or its author, with others. Only a small per cent 
quote a table of contents, and no more than half include any other 
quoted material. Special features, such as bibliographical material, 
problems or questions for study, summaries, etc., are seldom men- 
tioned. 

A more serious fault with the present-day book review is the lack 
of critical evaluation. The vast majority of reviewers point out the 
good qualities of books, or, very frequently, make general statements 
in praise of them, to the utter neglect of any deficiencies. The usual 
method of evaluation appears to consist merely of stating that the 
book makes a contribution to its field. It appears justifiable to say that 
reviews are failing to render a needed service to their readers. Teachers 
could select their textbooks more intelligently if reviewers would dis- 
tinguish between valuable books and those of doubtful merit. And per- 
haps some amateur writers would be discouraged from rushing into 
print, if reviewers had the courage, or took the pains, to criticize in a 
meaningful rather than a stereotyped manner. 


It should be recognized that editors have difficulties in se- 
curing the kind of evaluation suggested by Monroe and Hull. 
Cattell discusses such problems in terms of cost of space de- 
voted to review departments, objective and impartial selection 
of a limited number of books meriting review (some books 
may be of interest to only three persons—author, publisher, 
and reviewer), difficulty in securing critical signed reviews, 
promptness on the part of reviewers, and editing of reviews 
submitted. 

At its annual meeting in February, 1930, the Educational 
Press Association of America approved a report of its commit- 


tee on policies with respect to book notices and reviews. The 
nine recommendations are as follows: 1° 


1. The reviewing policy of an educational magazine should be con- 
sistent with the purposes and policies of the magazine as a whole. 

2. Only books and other publications pertaining to the field the 
magazine seeks to cover should be accepted for notice or review. 

3. Reviewing should be recognized as requiring both appropriate 
professional equipment and special skill in exposition and criticism. 

15 J. McKeen Cattell, “Book Reviews in Science,” Science, LXIX (February 22, 
1929), 220-2. 


16 “Policies as to Reviews in Educational Journals,” Educational Method, IX (Apri, 
1930), 437-8. ; ; 
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The practice of inviting reviews from educators in general is, therefore, 
not to be commended. 

4. Only books deserving positive praise or dispraise should be re- 
viewed. Others should be merely noticed. 

5. The practice of making laudatory comments on all books received 
is especially to be condemned. 

6. Reviews should be regarded as essentially editorial in character 
and should support the educational position and point of view of the 
magazine in which they appear. 

7. Reviews and notices of books should be regarded as equal in im- 
portance with other departments of the magazine. In accordance with 
this principle, they should be printed in the regular body type of the 
journal, not reduced. 

3. It is the duty of. each educational journal to inform publishers 
as to what books and other publications will be reviewed or noticed. 

9. It is proposed that the Educational Press Association take steps 
to prepare a standard scale of excellence in reviews to serve as a 
guide to editors and reviewers. 


Books by Gard and Jones assist in the preparation of re- 
views or evaluations of books." Gard’s discussion of the 
reviewer’s aims and methods, editors’ opinions on reviews, and 
types of reviews is applicable to the field of education, al- 
though his major concern is the literary review. 

Periodical literature. No systematic attempt in published 
form has been made to evaluate articles which appear in pro- 
fessional journals, although a number of digests, indexes, and 
abstracts (listed in Chapter III) serve as guides to the periodi- 
cal literature. However, in most instances the aim of such 
library aids has been completeness rather than evaluation or _ 
critical analysis, for example, Psychological Abstracts and So- 
cial Science Abstracts (discontinued in 1932). The Teachers 
Journal and Abstract (discontinued in 1932), Educational Ab- 
stracts, the Education Digest, and the Loyola Digest, as well 
as the bibliographical departments of a number of current 
periodicals, do and have attempted to exercise discrimination 
in selecting the articles to be reviewed, although the resulting 


17 Wayne Gard, Book Reviewing. New York: Alfred A. Knopf, 1927. Pp. xii + 150 
Llewellyn Jones, How to Criticize Books. New York: W. W. Norton and Co., 1928 


Pp. 190. 
‘Also see W. F. Spahr and R. J. Swenson, Methods and Status of Scientific Research, 


Chapter IFI. New York: Harper and Bros., 2930- 
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abstracts are of the summary type with no attempt at critical 
analysis or evaluation. Among the more serious attempts on 
the part of journals to provide a selected annotated bibliog- 
raphy of current research articles is the Department of 
Research Abstracts and Bibliographies of the Journal of 
` Educational Research. In its summaries of investigational 
literature, the Review of Educational Research analyzes arti- 
cles as well as books, monographs, and bulletins, although it is 
in no sense a month-by-month review of research of the 
preceding month or two. In fact, the editors of each issue 
select the investigations involved from approximately the 
preceding three-year period. Many of the “Selected Refer- 
ences” in the School Review and Elementary School Journal 
are from periodicals. 

As in the case of book reviews, objection has been made to 
the non-critical character of many of the articles in educational 
periodicals. One writer especially emphasizes the non-critical 
type of writing on higher education found in current profes- 
sional journals, and suggests that the reader’s inclination is to 
turn from such subjective and descriptive material to the 
factual literature of education: ** 


The temptation is strong tu turn to the increasing volume of edu- 
cational literature purporting to be the result of rigidly scientific in- 
vestigation, presented with mathematical accuracy and embellished 
with a redundancy of statistical refinements. But the curious absence 
of thought in much of this stuff has been often the subject of com- 
ment. 

The fact that most of the current criticism of higher education ap- 
pears in the “lay magazines” is not the result of accident. Whether 
from timidity or from preoccupation with pseudo-science, the bulk of 
the educational periodicals are inhospitable to critical contributions. 
Let us have more critics, lest we forget how to treat them honestly and 
capitalize their services. To be sure, the man who applies the acid 
sometimes burns his own fingers; but the corrosive agent is necessary 
for the reduction of dross. No professional literature can progress with- 
out liberal and continuous infusions of cauterizing criticism. 


Although there is much truth in the foregoing statement, the 


18 Rococo Facade, “Acid in the Literature of Education,” School and Society, XXXIV 
{September 19, 1931), 401-3. 
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material assembled in this chapter is evidence of numerous 
careful attempts to provide critical evaluations of educational 
literature and procedure. The type of reflective thinking and 
analysis suggested in the preceding quotation must not be taken 
as a substitute for factual investigation and experimentation, 
but as complementary and supplementary to (really a com- 
ponent part of) scientific studies of educational problems. 
Students of objective methods will agree that at intervals criti- 
cal analyses and evaluations, even though subjective in part, 
are needed to synthesize the progress made in given fields and 
to project probable future lines of development and study. 
Numerous illustrations of this type of thinking dealing with 
authoritative statements of accomplishment, analyses of educa- 
tional trends, critiques of research, and educational prophecies 
are given in Chapter II. 

A check-list of questions, used by the editorial board of the 
Journal of Educational Research to encourage better writing, 
is suggestive as an instrument of evaluation, whether applied 
to one’s own work or to the writing of others. 


t. Have you given a clear and concise statement of the problem 


that you have chosen to discuss? 
2. Have you explained, in a few words, the importance of the 


problem? 

3. Is the method of research employed in your investigation ap- 
propriate to the problem studied? 

4. Is the sample used in your investigation adequate? (Most ex- 
perimental research involves in one form or another the matter of 
sampling. Conclusions, to be of general value, should be based upon 
adequate samples.) 

5. Have you given an adequate description of the subjects used 
in your investigation? (If the results of your investigation are to be of 
value to others, your report should contain a concise, objective descrip- 
tion of the subjects used.) 

6. Have you supplied an adequate description of the materials, 
processes, or procedures under investigation? (It is not ordinarily suf- 
ficient merely to indicate these by name, as: the project method versus 
the traditional recitation; supervised study versus unsupervised study; 
etc. The thing under investigation will need careful and objective defi- 
nition.) 


7. Have you given a careful statement of the procedure followed 


692 EDUCATIONAL RESEARCH 


in your investigation, so that any one else might repeat the experiment 
from the description given? 

8. Have you provided adequate controls for such factors as may 
materially affect the results of your experiment? 

9. Have you validated the data-gathering devices used in your 
investigation? (Tests, questionnaires, observations, interviews, etc.) 

to. Are the tabular and graphical means of analysis used in your 
investigation the appropriate ones? (Check tables for numbering, titles, 
etc. Each table should be clear and complete in itself.) 

11, Are the statistical methods employed in your investigation ap- 
plicable to the materials in hand? Are the computations correct? 
(Many errors occur in statistical computations.) 

12. Is there a brief summary of your findings at the end of the 
paper? (These findings can frequently be summarized in one-two-three, 
serial order.) 

13. Are your findings and conclusions supported by the data pre- 
sented? (Articles frequently carry conclusions not at all supported by 
the data presented.) 

14. Have you read your manuscript for spelling, punctuation, and 
grammatical construction? 

15. Have you checked the footnotes and the bibliography for ac- 
curacy and completeness? (The mere listing of references by title and 
author is not sufficient.) 


Evaluation of theses. Statements made at the beginning of 
the present chapter suggest that preceding chapters of this 
book contain information of value in the formulation of cri- 
teria for use in the evaluation of educational research, including 
graduate theses. A useful summary of points to be considered 
in the analysis of graduate theses has been based on the judg- 
ments of sixty-one leaders in educational research, distributed 
among forty-two higher institutions of learning, state depart- 
ments of education, and national offices: 1° 
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(From the viewpoint of the graduate student 
and his adviser) 
I. Properly delimits the problem (Value ) 
(1) Defines the problem attacked adequately and (2) places it 
definitely in the particular field of research in which the 
student is working, 


19F, L. Whitney, “The Evaluation of Educational Research,” School and Society, 
XXXI (March 1, 1930), 289-90. 


Ii. 


IV. 
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Uses appropriate methods (Value ) 

Employs (1) the historical, (2) experimental or (3) predictive 
types of investigation carried perhaps on to the level of (4) 
philosophical analysis, each applied specifically to the field of 
endeavor engaged in. 


Exhibits originality (Value ) 

(1) The problem attacked is either new or distinctive; (2) the 
techniques used are adequate to the conditions of the study; 
(3) the method of reporting illustrates a high type of ability 
in discussion and interpretation of material presented. 


Uses proper thinking methods (Value ) 

(1) Determines actual ultimate aim, (2) states title correctly, 
(3) carefully states the problem, (4) discovers the thesis, 
(5) insures continuity with previous research by checking 
the findings of all other workers in the field entered, (6) ar- 
ranges procedures with the ultimate objective in view, (7) 
selects the best methods for each item of procedure, (8) sets 
up the form of report logically and psychologically as chap- 
ter heads and section titles, (9) maintains an open-minded 
and critical attitude at every stage of progress through the 
research, (10) provides for continuity of research onto the 
next level of reflective thought by the next research worker. 


Presents a worth-while contribution to education (Value ` ) 
The study results (1) in generalizations on a higher level of sci- 
entific thought, which takes the form of (a) hypotheses, (b) 
theories, (c) principles, (d) laws, (e) standards, (f) historical 
trends or (g) prognoses—all presented in terms of definite mean- 
ing; (2) in new data presented or old information and materials 
organized into new forms and relationships with adequate inter- 
pretations; (3) in better methods in educational research or the 
development of new techniques or the validation of old tech- 
niques, or their use in new fields; (4) in new tools of precision 
for use in handling educational concepts; (5) in the application 
of findings to concrete educational problems. 


Wilson reports an attempt to classify research to indicate 
relative values and applies his standards to the studies selected 
for review in two summaries of arithmetic investigations. The 
classification follows: *° 


20G. M. Wilson, “Research: Suggested Standards for Summarizing and Reporting 


Applied to Two Recent Summaries 


of Studies in Arithmetic,” Journal of Educational 


Research, XXVIII (November, 1934), 187-94- 
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1. A Major Research. There should be an important problem, a 
profitable technique, and data sufficient to leave little doubt as to the 
correctness of the conclusion. In a methods research there should be 
several rooms and appropriate grades, let us say not less than twenty 
rooms for experimental and twenty for control group, carried over a 
sufficient period of time. In case the findings are inconclusive, the ex- 
periment should be repeated. Case studies or intensive room studies 
could be accepted on the basis of less numbers. For a conclusion with 
reference to curriculum material, there should be the objective check 
supplied by social usage, and the number of cases should run into the 
thousands, or be so widely spread as to leave little doubt as to the 
answer. 

Should a research be called a “major research” unless it gets the 
answer (positive or negative) to the problem set? At any rate, this is 
a question very much worth considering. 

2. Minor Research. The same general conditions as for a major 
research, except that any sizeable amount of data pointing to signifi- 
cant conclusions or even trends may be profitably reported and used 
as a problem for further study. 

3. Partial Report of a Large Study. Justified when the findings are 
running strongly in a single direction and point to a decided modifica- 
tion of current practice. 

4. Case Study. Due to the fact that so little has been reported 
under “case studies,” there is justification for report on a single case, 
provided it is done adequately. Twenty case studies in reading, arith- 
metic, spelling, or writing should no doubt be judged a major research, 
if relating to a single problem. 

5. Library Study. As the name implies, a significant study or 
summary from available sources. 

6. Discussion. Chiefly concerned with interpretation or points of 
view. Valuable in proportion to how fully all of the relevant facts 
have been kept in mind, and how judiciously they have been inter- 
preted. 


SELECTION AND EVALUATION OF TEXTBOOKS 


There are numerous occasions on which school workers must 
participate to some extent in the selection and evaluation of 
textbooks; therefore, it seems justifiable to devote a section 
of the present chapter to this problem. Comment has been 
made in this chapter, concerning location of reviews of text- 
books. The rating of textbooks as a normative-survey type of 
procedure was described in Chapter VIII. The availability of 
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an annual list ** of elementary and secondary textbooks and 
other reasonably complete information concerning methods of 
locating titles of textbooks were discussed in Chapter ITI, and 
may be supplemented by examining another helpful source.?? 
Fortunately a comprehensive analysis of the textbook problem, 
from a much broader point of view than the mere selection of 
textbooks, is available in the year-book of the National Society 
for the Study of Education, which includes a detailed score 
card for use in rating textbooks. The conclusions reached by the 
committee are as follows: ** 


1. The educational interest of the pupil must at all times be the 
primary consideration in appraising plans for making and selecting 
textbooks, 

2. The principle is cardinal that the selection of textbooks is the 
prerogative of the educational personnel of our schools. Hence the 
Committee urges that educational administrators should defend their 
exercise of this prerogative against the claims or the interference of 
others, whether they be publishers, members of school boards, poli- 
ticians, or other laymen. The Committee believes that the superintend- 
ent should take the final responsibility in recommending textbooks. 

3. School administrators should be aggressive in demanding high 
standards of practice on the part of selecting committees and on the 
part of publishers and their representatives. The Committee invites the 
Department of Superintendence of the National Education Association 
to frame standards of practice for the selection of textbooks and to 
place these standards in the hands of textbook publishers. 

4. The Committee likewise invites publishers to frame standards 
of practice for their field representatives and to place these standards 
in the hands of school administrators and selecting agencies. 

5. Unethical practices are so clearly detrimental to the public wel- 
fare that, in the judgment of the Committee, state or regional commis- 
sions representing the profession should be created which will be 
charged with the duty of investigating complaints, whether made 
against bookmen, schoolmen, or others, and with the further duty of 
making public all pertinent facts in the event that the complaints are 
justified. 


21M. E. Herriott and Others, “Textbooks of 1934,” Society for Curriculum Study 
News Bulletin, VI (January 15, 1935), 9-46. 
, “Textbooks of 1935,” Curriculum Journal, VIL (February, 1936), 15-43. 

22 Carter Alexander, op. cit., pp. 216-22. 

23 J. B. Edmonson and Others, The Textbook in American Education, Thirtieth 
Yearbook of the National Society for the Study of Education, Part II, pp. 305-8. 
Bloomington, Ill.: Public School Publishing Co., 1931. 
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6. The cost of textbooks is so negligible a part in the total cost of 
education, especially when account is taken of the value of good text- 
books, that further increase in cost is fully warranted where neces- 
sary to secure better instructional material for pupils. 

7. Free textbooks should be provided in all public schools in the 
interests of better educational opportunities for the children and of 
economy to the general public. 

8. State adoption of textbooks often gives rise to questionable 
practices in connection with the selection and prescription of the texts. 
This Committee believes that our profession should seek to modify 
existing legislation in such a way as to eliminate these practices. The 
Committee believes, indeed, that the state is not a desirable unit for 
textbook adoption, that, on the contrary, the unit for adoption should 
be the local unit of schoool administration and supervision. 

9. State publication of textbooks is unwise, uneconomical, and 
educationally unsound. Our profession should continue to resist its 
extension, 

to. In making the two preceding statements with respect to state 
adoption and state publication, the Committee does not wish to be in- 
terpreted as condemning the principle of state uniformity in courses of 
study; the Committee recognizes that many facts, particularly with 
respect to the mobility of our population, point to the desirability of 
some degree of uniformity in subject-matter and in grade placement 
among the schools of a state or even of a larger area than the state. 
Reasonable uniformity in subject-matter and grade placement, particu- 
larly in the elementary school, would greatly facilitate the provision of 
adequate textbooks and need not destroy the values inherent in di- 
versity of textbook approach and treatment. 

tr, Publishers should feel obligated to refuse to publish manu- 
scripts that do not meet high standards in textbook-making. 

12. There is much need for careful research on problems relating to 
the mechanical features of textbooks. Publishers should be encouraged 
to carry forward such research, 

13. The critical trial of instructional materials in classrooms be- 
fore publication in textbook form is commended. 

14. Publishers have real cause for protest against some of the prac- 
tices of school administrators as listed in this Yearbook in the chapters 
entitled “Current Practices in Selecting Textbooks for the Elementary 
Schools” and “The Problems of Publishers in Making and Marketing 
Textbooks.” In particular, the Committee believes that the publishers 
have cause for protest and grounds for legal action against those 
schoolmen who engage in the practice of reproducing copyrighted ma- 
terial without securing the consent of the publishers. 


Se The use of the plan of secret committees in textbook selection 
is not good educational practice. 
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x6, For authors who are in educational work to use their positions 
to secure adoptions of their textbooks is likely to invite criticism en- 
dangering the good repute of our profession. 

17. The use of a score card for the evaluation of textbooks has 
certain obvious advantages in directing attention systematically to vari- 
ous items that should receive consideration. On the other hand, seem- 
ing numerical precision may be misleading, in that the qualitative 
whole is seldom to be measured by the sum of its quantitative parts. 
Score cards devised in terms of a particular book should not be used in 
the selection of textbooks. 

18, Teachers, as the users of textbooks, should have a voice in their 
selection, but the Committee calls attention to the fact that effective 
participation on the part of the teachers requires special competence. 

19. The choice of textbooks is so important an educational task 
that the study of-approved methods and standards for selection should 
be emphasized in the professional preparation of teachers. 

20. Supervisors of teachers in service and instructors in profes- 
sional schools should show teachers how to follow and also to supple- 
ment the book intelligently with respect to both content and method. 

21. American publishers are entitled to much praise for producing 
textbooks that represent high qualities of book-making and generally 
also of qualities of content and organization not found in textbooks of 
other countries. The Committee believes that too much praise cannot 
be given to the reputable publishers of the United States for the me- 
ticulous care with which manuscripts are examined and with which 
errors in form and content are detected and corrected. 

22. The Committee lends its full endorsement to the conclusion 
that the standards of practice in the selection of textbooks not only 
are higher to-day than formerly but also are relatively higher than 
those which prevail in many other lines of business. This conclusion is 
in agreement with the returns reported in the chapter entitled “The 
Ethics of Marketing and Selecting Textbooks.” The Committee further 
believes that this trend is full of encouragement and that it points to 
the possibility of an early solution of certain problems in making and 
marketing of textbooks. 


RATING COURSES OF STUDY 


In view of current interest in curriculum problems, including 
the writing and evaluation of courses of study, no justification 
of this section of the present chapter seems necessary. As in the 
case of rating textbooks, the technique employed in evaluating 
courses of study is normative-survev in character and was dis- 
cussed in Chapter VIII. 
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An elaborate attempt has been made at Teachers College, 
Columbia University, to evaluate elementary courses of study 
with the aid of judges and scales.** The investigation involved 
9,000 courses of study. The lists of “excellent” and “very good” 
courses of study provided have been of value to school people 
who must depend, at least in part, on the best current practice. 
This work of evaluation has been continued since the publica- 
tion of the first bulletin of the Columbia Bureau of Curriculum 
Research,” 

Periodical analyses and critiques of current courses of study 
such as those prepared by Harap are valuable and suggest a 
type of evaluation which may be done more frequently. His 
conclusions concerning 242 courses of study issued in 1928- 
1929 follow: ° 


If I were asked after examining 242 of the most recent courses of 
study, to give the essential characteristics of a modern course, I 
should include the following: (1) It states specific objectives of school 
work, (2) It includes a procedure or a statement of directions for car- 
rying on school activities. (3) It includes active experiences as well as 
mental experiences. (4) The whole is subdivided into small parts, called 
units for convenience. A unit here, is a small subdivision of a course 
of study which has unity or completeness, and the goal of which is an 
objective or group of related objectives. 


A characterization by the same author of courses of study 
published during 1932-1934 is as follows: ° 


24 Florence B. Stratemeyer and Herbert B. Broner, Rating Elementary School Courses 
of Study. Studies of the Bureau of Curriculum Research, Bulletin No. r. New York: 
Teachers College, Columbia University, 1926. Pp. xiv + 194. 

25H, B. Bruner, “Selected List of Courses of Study Published 1932-33,” Society for 
Curriculum Study News Bulletin, IV (November 27, 1933), 2-7. 

, “Outstanding Courses of Study of 1934,” Society of Curriculum Study 
News Bulletin, VI (January 15, 1935), 2-8. 

, and Mabel V. Cassell, “Annual List of Outstanding Courses of Study,” 
Curriculum Journal, VI (December 2, 1935), 1-9. 

26 Henry Harap, “A Critique of Public School Courses of Study 1928-1929,” Journal 
of Educational Research, XXI (February, 1930), 109-19. 

——, and Alice J. Bayne, “A Critical Survey of Public School Courses of Study 
Published 1929 to 1931,” Journal vf Educational Research, XXVI (September, October, 
1932). 46-55, 105-9. 

27 Henry Harap, “A Survey of Courses of Study Published in the Last Two Years,” 
(1932-34), Journal of Educational Research, XXVIII (May, 1935), 641-56. 

See also E. M. Draper, Principles and Techniques of Curriculum Making. New York: 
D. Appleton-Century Co., 1936. Pp. xvi + 875. 
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There is evidence of an increasing attention to the theoretical foun- 
dation of programs of curriculum revision. The importance of setting 
up meaningful goals to give direction to pupil activities is being in- 
creasingly recognized. A large proportion of courses appear to be 
socially useful or functional in character. To a greater degree units of 
work are being based on social experiences and lifelike interest centers. 
Although the number of published integrated courses is still small, in- 
tegration has emerged in the last two years as a significant curriculum 
trend. Typographically, the bulletins continue to improve, and the 
same may be said for their literary form. Sections of the country 
which have hitherto lagged in curriculum revision are beginning to 
show a lively interest. 

What of the future of curriculum making? The abundant output of 
courses of study, the general practice of offering courses in curriculum 
making, the common emphasis on the curriculum in programs of pro- 
fessional meetings, the recent emergence of interest in statewide pro- 
grams of curriculum revision, the uninterrupted flow of published 
treatises, the sustained interest in the curriculum revision movement, 
and the recently accelerated drive to revise the curriculum of the sec- 
ondary schools—these are signs of promise of continued progress in 
the curriculum of the American School. 


Summaries and bibliographies of attempts to evaluate or rate 
courses of study and textbooks are available.”* 

Concluding statement. As is apparent from a reading of 
the first paragraph of this chapter, any comprehensive attempt 
to summarize criteria or standards for evaluation of educational 
research and the techniques employed would mean repetition 
of many principles developed in other chapters of this book. 
Therefore, the immediately preceding pages have been limited 
to a consideration of criteria for evaluating educational liter- 
ature in the form of professional books and monographs, peri- 
odical articles, graduate theses, textbooks, and courses of study. 

Although it is recognized that the educational investigator 
and field worker who contribute in any significant way to the 
discovery of truth and its application to educational practice 
must evolve critical standards of evaluation and incorporate 


28 Henry Harap, “Evaluation of Curricula and Texts,” in “The Curriculum,” Review 


of Educational Research, I (January, 1931), 43-5- 7 A 
, “Evaluation of Courses of Study and Textbooks,” in “The Curriculum,” 


Review of Educational Research, IV (April, 1934), 194-8. 
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them in their own professional and scientific background, it 
will be conceded that assistance and guidance are desirable in 
the formulation of such criteria. To provide a maximum amount 
of this assistance to the younger workers in education, with 
due economy of time and space and without seeming to dictate 
arbitrary standards of evaluation, is the purpose of this chap- 
ter and of the numerous bibliographical references cited. 


PROBLEMS AND EXERCISES 


1, Select a completed study of interest to you, and evaluate it in 
terms of standards developed in this and other chapters, covering such 
matters as: 

(a) Statement of the problem 

(b) Survey of the related literature 
(c) Procedures for collecting data 

(d) Analysis and interpretation of data 
(e) Conclusions and generalizations 

2. Choose a professional book in education which has appeared 
within two years of undertaking this assignment, and locate three 
reviews of it, comparing them critically in terms of the principles of 
effective reviewing stated in this chapter or in some authoritative 
source. 


SELECTED REFERENCES 


* armack, J. C. Research and Thesis Writing, Chapter XI. Boston: Hough- 
ton Mifflin Co., 1930. 

Asupavcn, E. J. “Scientific Thinking in Education,” Journal of Educa- 
tional Research, XXV (April-May, 1932), 300-1. 

Baxer, H. J. “The Tower of Babel,” Journal of Educational Research, 
XXII (June, 1930), 60-2. Urges that educational workers and teachers 
be not overwhelmed by new and contradictory theories, philosophies, 
and fads. 

Barr, A. S. “Expressing the Scientific Spirit Through Better Supervision,” 
Journal of Educational Research, XXIX (September, 1935), 47-52. 

“The Scientific Method and Classroom Supervision,” Journal of 
Educational Research, XXIV (September, 1931), 141-3. 

Brackuursr, J. H. “Do We Measure in Education?” Journal of Educa- 
tional Research, XXVII (December, 1933), 273-6. 

* Bone, B. H. “Where Does One Go for Fundamental Assumptions in Edu- 
cation?” Educational Administration and Supervision, XIV (Septem- 
ber, 1928), 361-70. Emphasizes the philosophical method as of major 
importance in the solution of educational problems. 

* Breen, F. S. “Is a Science of Education Possible?” Scientific Monthly, 
XXXIX (Hecember, 1934), 530-5. Urges that education be placed 
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on a scientific basis, thus escaping the stagnation of traditionalism 
and the uncertainties of radicalism. ` 

+ Bricos, T, H. “A Philosophy of Secondary Education Today,” Teachers 
College Record, XXXVI (April, 1935), 593-603. 

* ——— “Jeremiah Was Right,” Teachers College Record, XXXII (May, 
1931), 679-95. A criticism of secondary and higher education and of 
educational investigations as now conducted. 

* Brim, O. G. “Research That Prevents Research,” Journal of Educational 
Research, XXII (October, 1930), 161-71. Urges the application of re- 
search methods to more difficult and fundamental problems, 

Brooks, Fowler D. “Criteria of Educational Research,” School and So- 
ciety, XVIII (December, 1923), 724-9. Discuss standards by which 
any piece or research is evaluated; the exact delimitation of the prob- 
lem; its analysis into constituent elements of subordinate problems; 
its historical background; selection and use of methods of investiga- 
tion; collection and evaluation of data; interpretations of data; reach- 
ing a solution or conclusion. Points out current errors and deficiencies 
made in following these procedures, 

Brown, C. H. “A Hazard to Research,” Journal of Higher Education, Il 
(November, 1931), 420-6. 

*Bucxrncuam, B. R. “The Philosophy and Organization of Research,” 
School and Society, XXIX (June 15, 1929), 755-64. Discusses re- 
search as a method, as a means of discovering new truth, and as a 
contribution, 

* Burton, W. H. Introduction to Education, Part II. New York: D. Apple- 
ton-Century Co., 1934. ` 

Carpwett, O. W. “Science—Truth or Propaganda,” School and Society, 
XXXVII (February 11, 1933), 169-74. 

Cuarters, W. W. “Fads,” Educational Research Bulletin, VIL (October 
31, 1928), 326-7. Points out that national interest in educational 
ideas has a span of three or four years—project method from 1917 
to 1920, curriculum from 1924 to 1927, and next character education 
seemed popular. 

“Measures of Research,” Educational Research Bulletin, XI (Oc- 

tober 12, 1932), 309-20. Analyzes the work of the Bureau of Educa: 

tional Research at Ohio State University. 

“Rating Scale for Class Reports,” Journal of Higher Education, I 
(June, 1930), 341-3. 

Crarke, E. L. The Art of Straight Thinking. New York: D. Appleton and 
Co., 1929. Pp. xi + 470. 

Cremers, F. O, “Evaluating Research Ideas,” Scientific Monthly, XXII 
(May, 1926), 441-4. Treats of the possible contributions of research 
to society, and the consequent responsibility on the research person- 
nel for the selection of worth-while problems and for scientific pro- 
cedure throughout an investigation. Discusses some criteria of a good 
problem. 

Corey, S. M. “We Need Less Logical and More Empirical Evidence,” 
School and Society, XXXIII (June 20, 1931), 831-2. 

* Counts, G. S. “Criteria for Judging a Philosophy of Education,” School 
and Society, XXX (July 27, 1929), 103-7- States that a good philos- 
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ophy should be systematically empirical in foundations, comprehensive 
in outlook, consistent in its several departments, practicable in its 
provisions, and satisfying to its adherents. 

Courtis, S. A. “Education—A Pseudo-Science,” Journal of Educational 
Research, XVII (February, 1928), 130-2. 

Courtis, S. A, “The Evolution of Individualization: An Interpretation,” 
Educational Method, XV (March, 1936), 291-8. 

“Significant Criteria for the Appraisal of the Validity of Con- 
temporary Educational Philosophy,” Educational Method, IX (No- 
vember, 1929), 66-72. 

Cox, P. W. L. “The Research Racket,” Junior-Senior High-School Clear- 
ing House, VII (January, 1933), 260-5. A criticism of questionable 
practices in educational investigation. 

“Twenty Theses Regarding Educational Research,” Educational 

Service, III (July, 1931), 14-5. 

* Coxx, W. W., and Others. The Grouping of Pupils. Thirty-fifth Yearbook 

` of the National Society for the Study of Education, Part I. Bloom- 
ington, Ill.: Public School Publishing Co., 1936. Pp. x + 315. 

DEMIASHKEVICH, M. J. “The Science of Education,” Phi Delta Kappan, 
XIV (April, 1932), 184-90. 

Dickey, J. W. “Statistical Ability Necessary to Read Educational Jour- 
nals,” Journal of Educational Psychology, XXVII (February, 1936), 
149-54. 

Ecrnton, D. P. “Tests and Measures of Satisfactory Growth,” Educa- 
tional Method, XV (December, 1935), 145-9. 

Encetuart, M. D. “Educational Research at the Crossroads,” American 
School Board Journal, LXXXVII (October, 1933), 15-7. 

Forry, J. L. “Progressives and Experimentalists as Allies,” Progressive 
Education, VII (May, 1930), 191-4. 

Fowrxes, J. G. Evaluating School Textbooks. New York: Silver, Bur- 
dett and Co., 1923. Pp. 34. 

Franzen, R. H., and Knicur, F. B. Textbook Selection. Baltimore: War- 
wick and York, 1922. Pp. 94. 

* Freeman, E. M. “Criteria for Judging a Science of Education,” School 

and Society, XXX (July 13, 1929), 48-52. 

Freeman, F. N. “An Analysis of the Basis of the Activity Curriculum,” 
Elementary School Journal, XXXV (May, 1935), 655-61. 

“Psychology as the Source of Fundamental Assumptions in Edu- 

cation,” Educational Administration and Supervision, XIV (September, 

1928), 371-7. Describes the psychological method as fundamental. 

“Teaching as an Applied Science,” Educational Method, 1X (May, 
1930), 448-55. 

Futter, Florence D. Scientific Evaluation of Textbooks, Boston: Hough- 
ton Mifflin Co., r928. Pp. xvi-+ 88. An experiment in cooperative 
evaluation of junior high-school mathematics texts. 

Gans, Roma. “Evaluation of the Work of the New School,” Education, 
LV (May, 1935), 530-4. Lists twelve questions to be answered. 

Goon, Carter V. “Educational Yearbooks and Annual Professional Pro- 
grams,” School and Society, XXXI (May 3, 1930), 604-6. Suggestions 
for the improvement ot year-books and scientific meetings. 


* 
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* GoopenoucH, Florence L., and Anperson, J. E. Experimental Child Study, 
pp. 50-4. New York: The Century Co., 1931. Criteria for evaluation 
of scientific work. 
* Gray, J. S. “Scientific Postulates for Educational Research,” Educational 
Administration and Supervision, XIX (January, 1933), 18-24. 
Gray, Olive. “A Teachers’ Convention—Diary of a Participant,” School 
and Society, XXXIV (December 12, 1931), 809-10. Criticizes lack of 
planning in the management of teachers’ convention programs. 
Harr-Quest, A. L. The Textbook: How to Use and Tudge It, Chapters 
IV, V. New York: The Macmillan Co., 1918. 
Harroc, Philip, and Ruopes, E. C. An Examination of Examinations. 
London: Macmillan and Co., Ltd., 1935. P. 81. 
Hissonc, Mary C., and Hissonc, Clyde. “The Scientific Movement and 
Progressive Education,” Education, LI (September, 1930), 1-9. Re- 
veals difficulties in measuring the results of the new schools, 
Hucues, W. H. “Two Educational Philosophies—Are They Compatible?” 
Nation’s Schools, XIV (December, 1934), 25-8. Stresses that the “new” 
school will provide for individual differences and that measurement 
will develop in the field of personality rather than be discarded. 
Huttrisn, H. G. “A Crucial Problem for Philosophy of Education,” 
Educational Administration and Supervision, XVI (April, 1930), 241-8. 
States that the crucial problem in a world of change is to work out 
an educational program so that the individual may leave the school 
with at least an emerging philosophy of life. 
Hunxuss, R. V. “Two Things Wrong with the Literature of School Ad- 
ministration,” American School Board Journal, LXXXIV (March, 
1032), 41, 100, 102. States that the literature in this field is confined 
almost exclusively to principles of school organization, and all but 
ignores operative technique or managerial art and smaller schools. 
Hunter, W. L. “Check List for Evaluating Research,” Industrial Edu- 
cation Magazine, XXXVI (January, 1934), 41. 
“Rating Scale for Research Reports,” Industrial Education Maga- 
zine, XX XVII (January, 1935), 51-2. 
Hurcutns, Robert M. “The American Educational System,” School 
Review, XLI (February, 1933), 95-100. 
Hutson, P. W. “Testing the Guidance Program,” Nation’s Schools, XV 
(June, 1935), 21-3- 
+ Jupp, C. H. “In Defense of American Secondary Schools,” School Review, 
XLI (April, 1933), 257-65- 
“The Development of the Science of Education in the United 
States,” International Education Review, IV (1032-1933), 475-96. 
Kerrey, T. L. “A Defense of Science in Education,” Harvard Teachers 
Record, I (1931), 123-30. 

Kirpatrrcx, W. H. “A Defense of Philosophy of Education,” Harvard 
Teachers Record, I (1931), 117-22- 

Kniss, F. R. “Selecting a Textbook in Tenth Grade World History,” 
Educational Administration ana Supervision, XXI (September, 1935), 


465-70. 
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Knupsen, C. W. “Experience versus Experimentation,” Peabody Journal 
of Education, VII (January, 1930), 242-3. Points out that certain new 
movements, such as the junior college and the junior high school, 
have been promoted by propaganda and tested indiscriminately at 
public expense rather than by subsidized experimentation. 

Lampert, A. C. “The Research Point of View in Education,” American 
School Board Journal, LXXXVI (March, 1933), 16. 

Laturop, Edith A. Aids in Book Selection for Secondary School Libraries 
Office of Education Pamphlet, No. 57, 1934. Washington: Office of 
Education, 1934. Pp. 28. 

Loomis, H. B. “Educational Philosophy: Change, Invariance, or Both?” 
School Review, XXXVIII (April, 1930), 256-62. 

McCrure, Worth, and Others. The Evaluation of Supervision. Fourth 
Yearbook of the Department of Supervisors and Directors of Instruc- 
tion. New York: Teachers College, Columbia University, 1931. Pp. 182. 

McConn, Max, “Educational Guidance: Progress Towards Scientific Pro- 
cedures,” School and Society, XL (October 27, 1934), 537-42. 

McConnerz, R. E. “Book Selection for a Teachers College Library,” 
Educational Administration and Supervision, XXI (October, 1035), 
551-4. 

Manvet, H. T. “Evaluating Present Educational Practice: An Illustra- 
tion,” Educational Administration and Supervision, XIV (Oc- 
tober, 1928), 493-8. Calls attention to the need for educational re- 
search and suggests some of the techniques available. 

Moxwett, C. R. The Selection of Textbooks. Boston: Houghton Mifflin 
Co., 1921. Pp. 138. 

Merso, I. R., and Waterman, I. R. “Evaluation of Textbook Materials 
in Handwriting,’ Elementary School Journal, XXXVI (November, 
1935), 204-10. 

* Monroe, W. S. “Hazards in the Measurement of Achievement,” School 
and Society, XLI (January 12, 1935), 48-52. 

* Morrison, H. C. “Sincerity in the Present Situation,” Educational Ad- 
ministration and Supervision, XX (April, 1934), 273-81. 

Noste, S. G. “The Beginnings of a Science of Education,” School and 
Society, XXXVIII (December 23, 1933), 835-6. 

Norris, O. O. “Some Major Postulates for a Philosophy of Education,” 
Educational Method, XI (May, 1932), 449-61. 

*Orara, P, T. “Begging the Question in the Name of Science,” Educational 

Administration and Supervision, XX (April, 1934), 241-63. 

“The Insignificance of the Statistically Significant,’ Educational 
Administration and Supervision, XX (May, 1934), 321-40. 

Parry, W. W., and Pater, W. I. “Improving Our Method of Selecting 
High-School Textbooks,” Journal of Educational Research, XXIV 
(June, 1931), 23-32. ; 

* Perters, C. C. “Educational Sociology as a Source of Fundamental As- 
sumptions in Education,” Educational Administration and Supervision, 
XIV (September, 1928), 385-92. Indicates Possibilities in the utiliza- 
tion of the methods of sociology in the solution of educational 
problems. 
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Prunty, Merle. “Evaluating Curriculum Reconstruction,” School Execu- 
tive, LV (February, 1036), 234-5, 239. 

* REISNER, E. H. “Science and Faith—Their Claims in Contemporary Edu- 
cation,” School and Society, XXXV (January 16, 1932), 78-83. 
“What Is Progressive Education?” Teachers College Record, XXXV 

(December, 1933), 192-201. 

REYMERT, M. L. “An Attempt at a Rating Scale for Psychologists,” 
Psychological Bulletin, XXX (January, 1933), 63-79. 

Roemer, Joseph. “Proposed Revisions of Secondary School Standards,” 
High-School Teacher, XI (May, 1935), 132-4. 

Rusk, R. D. “The Conflict of Science and Art,” Scientific Monthly, XXX 
(May, 1930), 458-64. Points out how science makes art more truly 
an accurate representation of nature. 

Sackett, E. B. “Is Research an Expense?” American School Board Jour- 
nal, LXXXVI (April, 1933), 19-20. 

Sears, J. B. “Technique and Procedure for Surveying the Curriculum,” 
California Journal of Secondary Education, X (January, 1935), 
123-31. Suggests ways for discovering weaknesses in the curriculum. 

Sxaccs, E. B. “The Limitations of Scientific Psychology as an Applied 
or Practical Science,” Psychological Review, XLI (November, 1934), 
572-6. 

Sxerres, A. G. “Research and Thinking,” Educational Administration 
and Supervision, XVII (April, 1931), 294-305. 

“What Is an Educational Philosopher?” School and Society, XXXII 
(July 12, 1930), 62-4. Includes definitions of educational philosophy 
and its function in a program of education. 

Stestvcer, Donald. “Standards for Evaluating Research Programs,” Jour- 
nal of Home Economics, XXVI (April, 1934), 206-11. 

Smr, F. W. “The Meetings of the Department of Superintendence,” 
School and Society, XXXIII (May 2, 1931), 591-2. Deplores the 
duplication of programs and the consequent waste of time and 
energy. + 

Smarts, V. T. “An Educational Philosophy,” Educational Administration 
and Supervision, XVI (February, 1930), 88-98. States a philosophy 
consisting of four propositions: that education is possible, that it is 
possible in varying degrees with different individuals, that education 
is democratic, and that education is living. 

THAvER, V. T. “When Science Goes Wrong in Education,” Journal of 
Educational Research, XXIV (June, 1931), 67-70. 

Tyter, R. W. “A Plea for Scientists in Education,” Educational Research 
Bulletin, IX (April 16, 1930), 222-3. Deplores | extreme and inexact 
statements in educational literature and emotional unwillingness to 
consider all the facts involved. 


* “Defining and Measuring the Objectives of Progressive Education,” 
Educational Record, XVII (January, 1936, Supplement No. 9), 78-85. 
* “Evaluation: A Challenge and an Opportunity to Progressive 


Education,” Educational Record, XVI (January, 1935), 121-31. 
“Evaluation: A Challenge to Progressive Education,” Educational 
Research Bulletin, XIV (January 16, 1935), 9-16. Emphasizes the 
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need for setting up appropriate objectives and adequate methods for 

attaining them, as well as measuring the extent to which the goals 

are achieved. 

“The Need for Adequate Data,” Educational Research Bulletin, 
VIII (December 4, 1929), 404-6. Lists information concerning stand- 
ard tests which publishers should furnish: method by which test 
items selected, correlations between test results and other criteria, 
reliability coefficients, examiner’s manual and scoring key, cost, and 
time required. 

WappeLL, C. W. “Some Basic Considerations in Evaluating the Activity 
School,” Educational Administration and Supervision, XVI (October, 
1930), 481-94. 

Warerman, I. R., and Mero, I. R. “A Plan of Procedure for the Evalua- 
tion of Textbooks in Reading,” Elementary School Journal, XXXV 
(May, 1935), 662-74. 

-“Eyaluation of Spelling Textbooks,” Elementary School Journal, 
XXXVI (September, 1935), 44-52- 

Wutrrtz, G. M. “The Improvement of Educational Research,” School 
and Society, XXVI (August 27, 1927), 249-59- Discusses the essentials 
of scientific work with illustrations of prevalent errors and short- 
comings involved in: the formulation of suitable problems; direct ob- 
servation; experimental technique; preliminary trial; use of the ques- 
tionnaire; measurement and statistics. 

“What Price Curriculum-Making?” School and Society, XXXI 
(March 15, 1930), 367-68. Points out. some of the dangers in cur- 
riculum making by committees and by amateurs. 

* Watney, F. L. Methods in Educational Research, Chapter VIII. New 
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CHAPTER XV 


THE TRAINING AND SUPERVISION OF RESEARCH WORKERS 


This chapter will discuss several problems of concern to ap- 
prentices in educational investigation, whether in the field or the 
graduate school, and to those who supervise research, whether 
directors of research bureaus, administrators, supervisors, 
or graduate professors. Among these topics are produc- 
tive scholarship, educational leadership, relation between ef- 
fective teaching and research, qualifications and characteristics 
of research workers, requirements for graduate degrees in 
education, codperative and codrdinated investigation, bureaus 
and departments of educational research, and other organiza- 
tions which play a part in the promotion and direction of 
research, 

Promotion of productive scholarship and utilization of 
research ability. The available evidence reveals that the level 
of scholarship and productivity, even in higher institutions, is 
lower than it should be. A questionnaire survey, conducted by 
the American Historical Association, indicates a lack of pro- 
ductivity in colleges, although they are placing more stress on 
research due to the method of training Doctors of Philosophy, 
the emphasis on the dissertation, the place of research in the 
sciences and in modern industrial life, and the belief that re- 
search involves a desirable method of education. The lack of 
productivity is attributed to the level of ability of candidates 
for the Ph.D. and the fact that the degree has become a com- 
mercialized teaching degree, the low social value placed on 
productive scholarship in the United States as compared with 
European countries, and a widespread belief that the research 
worker is not adequately rewarded for his efforts. Remedies for 
the comparatively low level of productive scholarship are sug- 
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gested as follows: the development of a greater interest in re- 
search, greater encouragement of Doctors of Philosophy who 
are capable of advanced work, more “weeding out” of candi- 
dates for the degree, more money for travel and publication, 
and greater recognition of scholarship in the professional world. 
These conclusions apply especially to the field of history, but 
they have a similar significance for the field of education.* 

One writer suggests thirteen reasons for the backwardness of 
scholars in doing the “big things” of the world and in pro- 
ductivity.” 


t. Most scholars will say that they are too busy to produce works 
beyond the completion of their daily round of tasks. 

2. Poverty is frequently a companion of scholarship. 

3. Laziness is often associated with scholarly ability. 

4. Stupidity ought never to be harnessed with scholarship, but un- 
fortunately it is. 

s. Poor writing and speaking may handicap persons who are 
otherwise creditable scholars. 

6. Small-mindedness may be a congenital limitation or a habit. 


7. Popularity may prevent potentially great personalities from 
becoming what they might. Š 

8. Formality and tradition often kill genius. 

9. Love of learning for learning’s sake is the rock on which some 
founder. 

to. Cowardice keeps many a scholar from amounting to something. 

11. Abnormal standards, too high or too low, prevent scholarly 
accomplishment. 

12, Conceit is often a fault of scholars. 

13. Lack of encouragement dampens the ardor of many an en- 


thusiastic scholar who might be a producer. 


A survey of 688 higher institutions concludes that if recom- 
mended or desirable practice is a valid indication of the prob- 
able trend of actual practice, the requirement of participation 


in research or “productivity” will obtain in from one third to 
more than one half of the colleges in the immediate future.® 


1 “Productive Scholarship,” School and Society, XXVI (October 22, 1927), 528. 
2A. S. Pearse, “Productive Scholarship,” School and Society, XXXI (April 12, 1930), 


493-6. 
3C. H. Thompson, “Some Administrative Requirements Governing the Appointment 


and Promotion of College Teachers,” School and Society, XXXVIII (September 9, 1933), 
347-52. 
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However, only five per cent of the college presidents actually 
required this productivity as a prerequisite for a salary in- 
crease, and only eleven per cent required this activity for 
promotion in rank. 

An analysis of the scholarship and productivity of the facul- 
ties of American teachers colleges and normal schools reveals 
that they bave only one third of the expected copyrighted pub- 
lications, one seventh of the expected representation in Amer- 
ican Meni of Science and in the American Botanical Society, 
and one half of the expected listing in the American Psycho- 
logical Register, as compared with universities and colleges in 
general. The pointed concluding statement of this report is 
as follows: * 


The facts pointed out in previous paragraphs should not be con- 
strued as an attempt to build up a destructive criticism of American 
teacher-training institutions. The writer gladly admits that there are 
definite limitations in the statistical comparisons made in this study. 
Obviously there are many other highly important factors to be con- 
sidered in determining the quality of the scholarship of faculties; it is 
also possible that there are matters of greater importance affecting the 
contribution which teachers colleges and normal schools will make to 
American education than the scholarship of its faculties. In spite of these 
limitations, however, it will be recognized that faculties of American 
teachers colleges and normal schools have serious scholastic short- 
comings which must be overcome in the very near future. Considera- 
tion of the far-reaching educational influence of these institutions, as 
pointed out in the opening sentences of this paper, demands the highest 
scholastic standards of its faculties. 


Faculty inbreeding, that is, the practice of selecting former 
students of an institution as faculty members, has been found 
dangerous for both the individual and the institution in terms 
of the probabilities of academic advancement, scholarly pro- 
ductivity, and outside professional recognition.® Such academic 


4P. V. Sangren, “The Scholarship of Faculties in American Teachers Colleges and 
Normal Schools,” School and Society, XXXIII (May 8, 1931), 642-4. 

Also see P. V. Sangren, “An Arraignment of Productivity,” Journal of Higher Educa- 
tion, II (February, 1931), 87-92. A criticism of the lack of productivity of American 
teachers colleges, based on an analysis of twenty-two journals. 

5 W. C. Eells and A. C. Cleveland, “Faculty Inbreeding,” Journal of Higher Educa- 
tion, VI (May, 1935), 261-9. 

„ “The Effects of Inbreeding,” Journal of Higher Education, VI (June, 1935), 


323-8. 
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inbreeding has increased markedly, especially during the decade 
from 1925-1935; more than one third of the faculty members 
in 219 institutions of higher learning in the United States 
possess one or more degrees from the schools in which they are 
employed. 

Adams quotes H. G. Wells who concluded that every pro- 
fessorial chair should hold three occupants and represent three 
important functions (research, teaching, and textbook prepa- 
ration).® 

However, it must be admitted that not all workers agree 
with the strong pleas for productivity and research voiced in 
preceding paragraphs. Certainly where specialization and con- 
centration render one narrow, usefulness in an applied field 
such as education is limited to that extent. Snedden maintains 
that there are four good reasons why scholars (defined as those 
who specialize in some field of science or art on a plane remote 
from the everyday work of elementary, secondary, or under- 
graduate collegiate schools) now exert very detrimental in- 
fluences on policy-making for public schools and undergraduate 
colleges: (1) the strong and even dominating personalities of 
many of these persons, (2) their influence on curriculum mak- 
ing through committees, textbooks, and addresses, (3) their 
subject-matter preoccupation, and (4) their failure to think 
of their specialties in terms of real, rather than imagined func- 
tional values.” 


The future of popular education, both at school and at college 
levels, must increasingly be determined by educators who can see and 
use the complex and elaborated sciences and esthetic arts of our day as 
means. Scholars we shall continue to have and to need—perhaps the 
more of them the better. But if popular education is to become more 
fully functional for culture or for any one of several kinds of material 
utility than it is now—and its present condition is far from happy or 
satisfying—the influence in its organization of those highly specialized 
minds here called scholars must be diminished and that of educators 


6 John Adams, “Professors and Books,” School and Society, XXXI (March 15, 1930), 


347-51. y 
7 David Snedden, “The Hurtful Influences of Scholars on Useful Educations,” School 


and Society, XXXI (February 1, 1930), 133-8. 
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who can use subjects of study as fruitful means towards ends of 
proved value be increased. 


Of special significance to all who have pursued graduate pro- 
grams of instruction is an urgent plea that the worker go 
forward in his given field of training and specialization, con- 
serving and extending the scholarship and research ability 
represented.’ 


A certain limited number of high-grade students obtain Ph.D. de- 
grees every year. At the moment of receiving the degree it is not un- 
likely that each of them is better posted concerning some aspect of 
his major field than any other man in the country. Yet, after taking 
their degrees, only a few Ph.D. students—and these are the wisest of 
them—pursue the topic which of all others they may pursue with the 
greatest likelihood of success. Much the same may be said of those 
who secure their Master’s degree. They have majored in a certain field 
and very likely have written a thesis in that field. No other area is for 
them so important as the one to which they have just given major 
attention. This particular topic offers them a greater chance than any 
other of securing recognition among their contemporaries. Differing in 
degree but not in kind is the major subject of the undergraduate 
student. Recent studies concerning the training of teachers have shown 
that too many of those who have fitted themselves abundantly to teach 
a certain subject enter some other line of service, thus dissipating their 
energies and incurring both for themselves and for society an unre- 
munerative expense. 


Fortunately there are a number of instances in which the 
young Ph.D has had the insight urged in the preceding para- 
graph. For purposes of illustration, certain titles of Doctors’ 
dissertations, together with the subsequent and present activi- 
ties of the recipients of the degrees, will be cited: 


1. “The Indexing of a Mental Characteristic”; teacher, author, and 
investigator in the field of statistics and factor analysis. 


2, “An Experimental Study of the Eye-Voice Span in Reading”; 
teacher of educational psychology and author of a number of analytical 


8B. R. Buckingham, “The Greatest Waste in Education,” School and Society, XXIV 
(November 27, 1926), 653-8. 

“The Greatest Waste in Education,” Journal of Educational Research, XIL (N 
1925), 311-4. 

V. L. Strickland, “Causes of Waste in Education,” School and Society, XXV (January 
1, 1927), 15%. 


‘ovember, 
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laboratory studies of the school subjects, utilizing extensively the tech- 
nique of photographing eye-movements. 

3. “Studies of Elementary-School Reading through Standardized 
Tests”; teacher, author, and investigator in the field of reading, in- 
cluding surveys, tests, and summaries. 

4. “The Administration of Secondary-School Units”; teacher, au- 
thor, investigator, survey worker, and editor in the secondary field, in- 
cluding the junior high school and junior college. 

5. “The Progress of English Grammar in American Schools to 
1850”; teacher, author, and investigator in the area concerned with the 
teaching of English, especially summaries of the factual literature. 

6. “A Study of School Postures and Desk Dimensions’; research 
adviser and consulting expert to a large manufacturer of school seating 
equipment. 

Of course, it is true that a change in the interests of the in- 
dividual or a shift of emphasis in the field of education may 
result in subsequent activity quite different from the thesis 
problem. For example, the young man who investigated “Arith- 
metic Tests and Studies in the Psychology of Arithmetic” 
twenty years ago has since turned to sociological and economic 
problems and comparative cultures as they affect education. 

Characteristics of scientists and leaders in education. The 
need for productive scholarship and research has been stated 
in preceding paragraphs. More definite information concerning 
the qualifications and characteristics of scholars, scientists, 
research workers, and leaders is of interest to those in training 
for such fields of activity and to those who supervise the in- 
structional and investigational programs involved. The qualifi- 
cations of a successful research worker have been listed as 
follows: ° 


1. Emotional factors—drive 
One cannot do successful research work unless (a) he has the 


urge that comes from curiosity, (b) he takes pleasure in self-expres- 
sion, in creation, (c) he feels that such work will be of real value 


either to himself or to others. 


9 National Committee on Research in Secondary Education, An Outline of Methods 
of Research with Suggestions for High-School Principals and Teachers, p. 11, Bureau of 


Education Bulletin, No. 24, 1926. Washington: Bureau of Education. i 
S. A. Courtis, “The Development of Ability in Research,” Studies in Education, pp. 


85-93. Yearbook Number XV of the National Society of College Teachers of Education. 
Chicago: University of Chicago Press, 1926. 
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2. 


Knowledge—efficiency 

No one is ready to do effective research until he has developed the 

following: 

(a) Ability to find quickly, to select, and to utilize the results of 
previous research 

(b) Knowledge of and facility in using approved experimental pro- 
cedures, inductive and deductive processes, logic and experi- 
mentation 

(c) Skill in using approved educational measuring instruments 

(d) Skill in analytical methods, statistical and graphic 

(e) Ability to generalize, to know when and how to make safe 
generalizations 

(f) Skill in verification involving criticism and prediction 

(g) Ability to select and organize the materials used in the inves- 
tigation and to report the methods and results 


Volition—control 
The volitional factors necessary may be outlined as follows: 
(a) Visible evidence 
(1) Courage 
(2) Persistence 
(3) Open-mindedness 
(4) Humility 
(5) Faith 
(b) General powers 
(1) Self-direction—independence in thought 
(2) Self-appraisal—power to know one’s strength and limita- 
tions 
(3) Self-control—power of self-control 
(c) Convictions 
(1) About nature—unity of law, evolution, cause and effect 
(2) About self—potentialities, social consciousness, control of 
self 
(3) About opportunity—reality and possibility of discovery of 
truth, possibilities of growth in self and progress of society 


Standards suggested for work in the natural sciences should 


be used to the extent possible in attacking educational prob- 
lems, although it is recognized that such criteria cannot be 
transferred bodily to the social sciences.2° 


1. Detachment 
2, Holds results as provisional 


10 J. M. Hughes, “The Educational Worker and Scientific Ideals,” American School 


Board Journal, LXXVIII (March, 1929), 60, 142. 
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3. Dislikes philosophical speculations 

4. Honest in reporting his work 

s. Carries his ideal of evidence, conception of proof, and attitude 
vf detachment into every situation and subject that concern him 

6. Refuses to take sides on issues amenable to investigation 

7. Unwilling to reason by analogy 

8. Differentiates between emotion and objective truth in his be- 
liefs 

9. Dissatisfied with knowledge that is not verifiable 


Kelley has sketched an interesting composite picture of the 
mental traits of men of science.’ He believes there are no 
exceptions to the first nine statements. Items 10-14 are common 
but not universal traits, items 15-22 are not uncommon, and 
items 23-27 are not unknown in men of science. 


1. The great man of science is industrious, shows great mental 
energy, and is persistent on the trail of a discovery. 

2. He questions authority, at least in the one line of his greatest 
achievement. 

3. He is apt at drawing inferences and is therefore ingenious in 
making hypotheses. 

4. His sense of logic is sound, so that he is perspicacious in making 
deductions. 

5. He is a keen observer of natural phenomena. 

6. He is dependent on observed facts. 

7. He is inventive in the matter of techniques. 

8. He is rich in his variety and number of hypotheses. 

9. He is not “inspirational” in his chosen field, i.e., his feet are 
always on the ground—his hypotheses are always amenable to some 
tests of a factual or observational sort. But he is inspirational in the 
sense that a vision not the common property of fellow-men urges 
him on. 

ro. He lacks personal attachment to a hypothesis—he will slay his: 
own mental offspring, or if he does not actually kill an erroneous 
hypothesis that he has given birth to, he will let it die from inattention. 

rr. He is disputatious. 

12. In his person much transfer of training takes place, for he 
adapts to one field a device, method, or hypothesis drawn from another 
field. 

13. He is versatile in his interests, and even in his native abilities. 

14. He is tolerant. 


11 T. L. Kelley, Scientific Method, pp. 182-83. Columbus, Ohio: Ohio State University 


Press, 1929. : 
— “Scientific Method, pp. 228-29. New York: The Macmillan Co., 1932. 
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15. In his person erroneous transfer takes place, for he reasons by 
analogy without warrant. 

16, He has a good memory. 

17. He has good mathematical ability, even though mathematics is 
not his major activity. 

18. He has excellent motor coérdination and manipulative ability. 

19. He has a thorough knowledge of antecedent work. 

20. He has a deep religious feeling. 

21. He is generous. 

22. He is precocious in his development. 

23. He is egotistical. 

24. He is modest. 

25. He has hobbies. 

26. He is not sociable. 

27. His ancestors are of more than ordinary distinction. 


In a helpful discussion of the qualities and aptitudes de- 
manded of the scholar, Spahr and Swenson emphasize consider- 
ation of: intellectual honesty; the influence of heredity, 
environment, and education; objectivity ; use of his own nature 
as a key in studying others; danger of magnifying recently 
discovered facts; preconceived conclusions and cherished hy- 
potheses ; emotional difficulties ; tendency to impair objectivity 
because of personal and pecuniary interests; optical illusions 
and peculiarities of hearing; confounding observation with in- 
ference; drawing important conclusions from insignificant 
tests; inadequate conceptions due to wide distribution of data 
in space and time; liking for one’s work; great ae ac- 
curacy ; and prudence, judgment, and good sense.’* 

A fuller statement of two of the most important character- 
istics of the scientist, desire to know the truth and honesty in 
reporting results, follows: + 


But what is a scientist? Not simply one who works the microscope, 
the telescope, or the retort; not simply one who works in the field of 
chemistry or physics or biology or astronomy. A scientist is one who is 
distinguished by two characteristics. First an insatiable desire to know 
the truth and, second, an absolute honesty in reporting what he finds. 


12 W. E. Spahr and R. J. Swenson, Methods and Status of Scientific Research, Chapter 
I. New York: Harper and Bros., 1930. 
13 “Research,” Educational Research Bulletin, VI (April 13, 1927), 164-5. 
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These will characterize Galileo and Newton, Faraday, Pasteur, and 
Michelson, They will also characterize Plato, Savonarola, Luther, 
Pestalozzi, and Thorndike. If we are to have a science of education, 
research must play a big part in the things we do, This means we must 
be scientists. We must have the characteristics of the scientist. 

Probably the vast majority of people fail to qualify on one or both 
of these characteristics. They have no insatiable desire to know the 
truth, Gossip is quite as satisfactory as fact. They prefer to take their 
opinions, ready made, from others, for it requires less effort than think- 
ing. Prejudices are more easily acquired than knowledge. It is easier 
to express an opinion than to pass judgment upon it; easier to repeat 
than to revise one’s position on the basis of more information. Also, 
it seems more consistent. 

The second characteristic—absolute honesty in reporting what we 
find—is equally important. Recently a professor of geography in a uni- 
versity was heard to say, “I don't care what the evidence shows, I 
don’t believe it.” On another occasion the dean of a graduate school 
remarked concerning some data reported in a Master’s investigation, 
“I don’t believe it, and if it is true, it should never be published.” An 
editorial, which appeared in an educational magazine a number of 
years ago, said this, regarding a certain investigation: “The study ought 
never to have been made. Having been made, it ought not to have been 
accepted as a Doctor's dissertation. Having been accepted, it ought 
never to have been published.” These statements are illustrative of the 
non-scientific attitude of mind which sometimes appears even among 
those whom we ordinarily suppose to be scientific. All of us have our 
prejudices. All of us have our limitations, especially outside of our 
own field, The true scientist, however, reports with absolute honesty 
his findings whether or not they agree with tradition, with the findings 
of his colleagues, or with the theories and philosophy which he holds. 
Truth is acknowledged and reported regardless of how it cuts across 
personal inclinations. 


With regard to the characteristic of absolute honesty in 
reporting results, a qualifying statement should be made. There 
may be times, audiences, and places more appropriate than 
others for reporting truth. It is questionable whether all known 
information concerning instruments of destruction, such as 
poison gas, should be broadcast to become the weapons of 
criminal individuals and groups or even of ruthless nations. It 
is said that a somewhat premature publication of a school sur- 
vey of the rural section of a given state, recommending con- 
solidation of schools, delayed the desired objective for many 
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years. Probably a program educating the public to the advan- 
tages of consolidation should have preceded or accompanied the 
release of the survey findings. Certainly some facts serve their 
purpose best when handled by those with the necessary inter- 
pretative background rather than when disseminated promiscu- 
ously where they may be misinterpreted or misused by ignorant 
or unscrupulous persons. 

Boyd has presented and illustrated what he considers the 
desirable qualifications for conducting research: youth, curi- 
osity, imagination, the experimental attitude, enthusiasm, pa- 
tience, persistence, faith, courage, common sense, honesty, and 
modesty.’* As illustrations of the accomplishments of youth, 
he cites the exploits of a large number of men under thirty 
years of age: Lindbergh, Columbus, Livingston, Stanley, Alex- 
ander the Great, Caesar, Hannibal, Charlemagne, Napoleon, 
Newton, James Watt, Eli Whitney, McCormick, and Westing- 
house. Of course, the important work of older and more mature 
men with almost a lifetime of experience and background 
must not be overlooked. The traits of intellectual curiosity and 
honesty have already been discussed in this chapter. Imagina- 
tion and common sense are involved in the formulation and 
testing of hypotheses, as discussed in Chapter IV. The char- 
acteristics of experimentation were outlined at some length in 
Chapter IX. Enthusiasm, patience, persistence, faith, and cour- 
age are represented in the work of Pasteur who took five years 
to find his remedy for hydrophobia, of Charles Goodyear who 
worked ten years before a lucky accident taught him how to 
vulcanize rubber, and of Faraday who spent ten years in his 
effort to “change magnetism into electricity.” As for courage, 
the fields of medicine and exploration have furnished the world 
with hundreds of illustrations too numerous to mention here. 
Although modesty is not a universal attribute of research 
workers, it is a trait definitely present in the makeup of many 
great scientists. 

The hardships and difficulties encountered by experimenters 

14T. A. Boyd, Research, Part IV. New York: D. Appleton-Century Co., 1935. 
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are almost proverbial.* The Curies made their long search for 
radium in a tumbledown shed that had been a dissecting room. 
Bell worked on his telephone in a Salem cellar and in a Boston 
attic, Morse made every foot of the insulated wire he used by 
winding cotton around bare wire. Having shown the world how 
to vulcanize rubber, Goodyear was in debt to the amount of 
two hundred thousand dollars at the time of his death. 

If not actual physical hardships, scientists frequently have 
encountered laughter, ridicule, and even persecution in 
some instances—Fulton’s steamboat, Stephenson’s locomotive, 
Franklin’s experiments with electricity, and the Wright 
brothers’ flying machine. Not even daring to reveal his 
thoughts and discoveries in conventional form, Leonardo da 
Vinci used reversed or mirror writing. Galileo was forced to 
“recant” with respect to his discoveries under threat of torture. 
However, it is true that the path of the research worker has 
been made much easier in late years through the good offices 
and support of such agencies as those cited in Chapter III and 
later in the present chapter. 

It should not be assumed from the foregoing analyses of the 
characteristics of scientists, scholars, and research workers that 
the whole of educational leadership consists of frequent par- 
ticipation in, and application of, investigational procedure, im- 
portant as this activity is. It has been suggested that, in 
addition to the work of the investigator, there are the impor- 
tant activities of those who devote their major energies to the 
functions of administration or interpretation of schocls,’® or to 
the development of “innovations” in educational procedure." 
Education needs the services of all of these workers. 

Alexander gives some rather practical advice to the young 
worker who is trying to decide between a field or an institu- 


15 Ibid., pp. 43, 53, 292-4. A 

16 J, E. Morgan, ‘Educational Interpretation and the Printed Word,” School and 
Society, XXX (July 20, 1929), 79-83- ‘ 
, “The Need for Educational Interpretation,” Journal of the National Educa- 
tion Association, XIX (February, 1930), 37-8- 

17 “Invention and Appraisal,” ‘Educational Methoa, 1X (December, 1929). 129-30. 
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tional career, the latter of course involving more emphasis on 
research and publication.’* 


A fair percentage of men consider going into one side and transfer- 
fering later to the other. Such transfers do sometimes occur, but they 
are not numerous or easy to make. After ten years in the field, a man 
is not regarded seriously by an institution, as compared with a young- 
ster who gives promise of research and publication abilities and will 
start at a low salary and grow up. Any field man called to an institu- 
tion usually owes the place to the fact that he has become so promi- 
nent and influential in his state or section that his outside contacts can 
be capitalized for bringing to the institution educ:iional opportunities 
that would not come to its regular inside faculty. It is not easy for in- 
stitutional men to go to good field positions, although we have some 
successful superintendents who changed from universities or normal 
schools. They had, however, so impressed the public mind by their 
activities that their being from an institution did not label them as 
“academic.” Some of them have returned to institutional work from 
the field. But the writer cannot recall a single such case where the man 
had not all along deliberately kept up the research ard publication so 
dear to institutional heads. For any man contemplating riding two 
horses this way, the best advice is to keep one foot securely on the 
field work horse, and the other foot on the institutional horse of re- 
search and publication, until he decides on which steed to put both feet. 
Otherwise, as regards a real career in educational work, he is sure to be 
indeed unhorsed. This means very different work for both feet, with 
the gaits and characteristics of the two horses always in mind. It is 
far more strenuous than riding a single steed. But riding two horses at 
the same time always means just that. 


Possibly the task of “riding the field and institutional 
horses,” as described by Alexander, is not hopelessly impossible. 
At least, knowledge of practical school problems and procedure 
and ability to work with people, on the one hand, and ac- 
quaintance with scientific method on the other hand are con- 
sidered important characteristics of a leader in educational 
research. 


In this plea for leadership in research I have asserted that due to 
the educational conditions of to-day, there is need of quantity pro- 


18 Carter Alexander, “Field Versus Institutional Careers,” Phi Delta Kappan, XI 
(February, 1929), 144-5; also Teachers College Record, XXX (April, 1929), 682-5. 

19B. R. Buckingham, “Leadership in Educational Research,” Journal of Educational 
Research, XV (April, 1927), 239-45. 
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duction in research. In order to produce the verified and scientifically 
defensible body of knowledge which is thus demanded, there must be 
leaders and followers. I have tried to give some of the qualities of 
leadership which seem to me to be desirable. The leader should have 
practical school experience. He should be able to enlist the support and 
win the confidence of teachers. He must be able to train workers. He 
must have the ability to analyze larger problems into their simpler 
elements and to combine selected parts of complex problems for the 
purpose of codperative research, He must be a scholar in his chosen 
field and must be able to state problems with authority. If he is a real 
leader he will have what I have called “extractive power.” He will be 
able to put his program over and to secure from people who would 
otherwise be indifferent the information he needs. He will be well in- 
formed as to the nature of scientific method and will know how to ap- 
ply such a method to educational data. On the other hand, he will know 
the limitations of strictly objective treatment. He will be expert in 
thinking as well as in figuring. He will follow the figures where they 
lead, impartially and without bias or prejudice, but he will know how to 
interpret them and to supplement them. Finally, he will study a ques- 
tion long enough and hard enough to bring out something more substan- 
tial than tentative conclusions or a timid turning over of his chosen 
question to the educational public for further study. 


In addition to the preceding analyses of scholars, research 
workers, and educational leaders, attention should be directed 
to similar references, used in another connection in Chapter 
III, and in the selected bibliography (under the names of Barr, 
Cooper, Edwards, Keys, Loofbourow, Noll, Snedden, Traxler, 
and Tyler) of the present chapter. Chapter I also has a bearing 
on the problem under discussion. At this time space may be 
taken for more specific mention of investigations of : vocational 
routes taken by 546 people in reaching certain educational posi- 
tions of leadership,” and leadership in curriculum making in 
168 school systems.** 

Relationship between teaching, administrative, supervi- 
sory, and research activities. It is assumed at this point that 
preceding portions of this volume have shown clearly the need 
for, and the contribution of, research. It is also assumed that 

oo Ruth Strang, “Vocational Paths to Certain Educational Positions,” School and 
Society, XXXVI (October 15. 1932), 508-12. 


21 Margaret A, Norton, “Leadership in Curriculum Building in 168 Large City School 
Systems,” School and Society, XXXIII (January 3, 1931), 17-20. 
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properly qualified teachers (at all levels of instruction), as 
well as administrators and supervisors, should have training 
in, and contribute to, problem solving in their respective areas 
of activity and responsibility. This statement is hardly debat- 
able, although the question of an appropriate balance between 
training in teaching or administrative duties, on the one hand, 
and problem-solving or investigational procedure on the other 
has been discussed frequently and vigorously in the educational 
literature. 

The point of view that virtually all research should be con- 
ducted in graduate schools, laboratories, or research bureaus 
is hardly defensible in terms of the desirability of the intel- 
lectual quickening of the million teachers in this country, 
ready application of the results of educational investigation, or 
discovery of important field problems pressing for solution. 
Certainly workers adequately prepared, through graduate 
school or other training programs, in their instructional, super- 
visory, or administrative fields and in problem-solving proce- 
dures appropriate to their areas of responsibility have a dual 
rôle and obligation of great importance and promise. It is 
recognized that in public schools and undergraduate colleges 
neavy teaching loads and the lack of facilities for research 
have been limiting factors, yet the individual with a keen in- 
tellectual curiosity will press forward in the search for truth 
in spite of obstacles. These dual responsibilities of teaching (or 
supervision, or administration) and of problem-solving seem 
not incompatible in the properly trained worker. There can be 
no doubt concerning the stimulating influence such a person 
brings io classroom or conference. It seems not unreasonable 
to suggest that teachers properly trained in methods of investi- 
gation should be able to direct more intelligently a problem- 
solving approach to intellectual growth on the part of pupils. 

The discussion found in Chapter Il and the extended lists of 
problems for study (in the appendix) suggest the wide range 
of topics available for investigation. A symposium on the par- 
ticipation of the field worker in educational research shows 


SUPERVISION OF RESEARCH WORKERS 723 


how the teacher, administrator, or supervisor may share in 
curriculum programs, instructional research, school surveys, 
and historical studies. In most such instances the field worker 
may add to his own professional understanding and training, 
while at the same time making some contribution to the solu- 
tion of important educational problems. 

The preceding paragraphs should not be interpreted to mean 
that all field workers are to be producers of research, without 
regard to their training and qualifications. In making these 
statements it is conceded that scientific experimentation is a 
difficult and complicated problem, much more so than the 
average lay or even professional worker realizes. It is recog- 
nized that not all teachers will rise to the level of making con- 
tributions to the science of education. This, of course, is also 
true of many so-called research workers. 

As pointed out in the preceding chapter and in the preface, 
the majority of field workers should be critical consumers of 
research rather than actual producers. There is really a ten- 
dency on the part of the educational practitioner to put en- 
tirely too much faith in the training researches of both teachers 
and research workers. Unfortunately, in too many instances, 
supervisors and professors seem to assume that they alone are 
to see problems, solve problems, and do the educational think- 
ing prerequisite to the successful management of school sys- 
tems. They hand down their solutions in the form of methods 
to be applied, textbooks to be used, courses of study to be 
followed, and policies for teachers and students to put into 


22 W. A. Brownell, “The Field Worker as a Consumer and Producer of Research,” 


Journal of Educational Research, XXIX (October, 1935), 144-8. 

H. R. Douglass, “Scientific Investigation of Instructional Problems,” Journal of 
Educational Research, XXIX (October, 1935), 130-3. 

H. G. Good, “The Possibilities of Historical Research,” Journal of Educational Re- 
search, XXIX (October, 1935), 148-53- 

J. Paul Leonard, “Teacher Participation in Recent State Curriculum Programs,” 
Journal of Educational Research, XXIX (October, 1935), 117-26. 

A. K. Loomis, “Needed Research in the Curriculum,” Journal of Educational Research, 
XXIX (October, 1935), 126-30. 

Jesse B. Sears, “The School Survey as a Means of Training Field Workers in 
Education,” Journal of Educational Research, XXIX (October, 1935), 138-43. 

Clifford Woody, “Stimulating Instructional Research in Michigan Schools,” Journal 
of Educational Research, XXIX (October, 1935), 937104. 
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operation. The teachers, then, only teach, acting neither as 
consumers nor producers of research. If one were to visit a 
classroom and find the teacher doing all of the thinking, enter- 
taining all of the points of view, and solving all the problems, 
the observer would characterize such a worker as inefficient. 
Judged from this point of view, much of the supervision, ad- 
ministration, and instruction of to-day is ineffective.** 

Another word of caution concerning wholesale encourage- 
ment and initiation of research among classroom teachers was 
written in answer to the suggestion that teacher-training insti- 
tutions use their staffs and facilities to provide guidance centers 
for teachers who were using their classrooms as research 
laboratories.* 


Such a plan is certainly worthy of serious consideration. However, 
it involves certain elements which cannot be overlooked. Vigorous sci- 
entific attack on educational problems means the expenditure of energy. 
If energy is to be available for this purpose, it must be provided by 
the public, which is to profit through research. At the present time 
boards of education and the public do not recognize the importance of 
continuous research in order to keep the educational system at a high 
level of efficiency. Research and certain very essential phases of 
central supervision are limited by the fact that the public is willing to 
have the schools move along traditional lines and makes little or no 
demand for innovation. Under such conditions teachers will be very 
slow to follow the plan suggested by the writer quoted, but public com- 
placency is certainly no adequate justification for inaction on the part 
of teachers. Perhaps the most promising suggestion is that concrete 
examples of school improvement supplied to the public by voluntary 
work on the part of teachers will do more than anything else to bring 
the kind of support necessary for a change in the whole situation. 


The reader who wishes to pursue this subject further will 
find numerous references in the chapter bibliography under 
the names of Alexander, Ayres, Barr, Carlson, Chamberlain, 
Chase, Courtis, Davidson, Ganoe, Goodman, Hepburn, Holmes, 
Jones, Jorgensen, Judd, Kearney, Kelley and Anderson, 
McMurry, Miller, Monroe, Newlon, Pace, Rogers, Sangren, 

23 A, S. Barr, “Research for Teachers,” Journal of Educational Research, XX (June, 
1929), 42-3. 


24 “Educational Research by Teachers,” Elementary School Journal, XXVI (April, 
1926), 564-6. 
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Trow, Weidemann and Rugh, Weld, Whipple, White, Woody, 
and Wykoff. 

Graduate training programs. It is to be expected that the 
majority of those who contribute in any significant way to the 
solution of educational problems have pursued some type of. 
graduate instruction. The preceding discussion of the relation 
between teaching and investigational activities has emphasized 
the need for training in both areas, which means that graduate 
training programs must provide the necessary instruction. As 
has been said, teachers, supervisors, and administrators need 
training in problem solving for their own intellectual stimula- 
tion and growth, to facilitate ready and rapid application of 
the results of research, and to aid in the solution of numerous 
and pressing field problems. For a consumer or a producer of 
research, training in the fundamental principles of investiga- 
tion is necessary, although more technical and more intensive 
specialization may be required for the producer. On the other 
hand, Doctors of Philosophy require instruction in sound edu- 
cational principles, since approximately three fourths of them 
enter teaching.2* Therefore, the authors have neither the in- 
clination nor the space to discuss the much debated question 
of whether there should be separate graduate degrees for pro- 
ficiency in teaching, or in administration, or in research. This 
question apparently resolves itself into one of proper balance 
in graduate programs in terms of the needs of a given worker 
rather than of the total absence of research training. Nor does 
the determination of a label (M.A., M.S., Ed.M., Ph.D., Ed.D.) 
for use at the completion of a given period of graduate instruc- 
tion seem especially significant for purposes of the present 


25 W.. W. Charters, “Editorial Comment,” Educational Research Bulletin, IX (February 
» 1930), 79, 
ae A “Graduate Schools and College Teaching,” Educational Research Bulletin, VIL 
(November 14, 1928), 348-9. 

M. E. Haggerty, “Occupational Destination of Ph.D. Recipients,” Educational Record, 
IX (October, 1928), 209-18. 

J. L. LaPoe, “The Significance of the Doctorate in the Preparation and Selection of 
College Teachers.” Unpublished Doctor’s thesis, Ohio State University, 1934- 

M. O. Wilson, “What the Chicago Doctors of Philosophy Are Doing,” School and 
Society, XXIX (June 22, 1929). 8ts-* 
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discussion, although a number of references in the chapter 
bibliography deal with this question. 

A vigorous statement of the desirability of retention of 
scientific training in graduate programs of instruction cautions 
_school workers against a return to speculation, authority, and 
practice in the determination of educational policies and pro- 
cedures.?° 


The scientific movement in education during the last thirty years has 
produced an entirely new type of literature, School teachers and ad- 
ministrators have a body of factual data on which to build practices 
which was almost completely lacking thirty years ago. A good begin- 
ning has been made, scarcely more than that, In contrast with the de- 
velopment in education through all the years when wishful thinkers 
have propounded their reforms, the progress of the past thirty years 
needs no defense. Yet, because the way of science is slow and tedious, 
impatient reformers now demand that we renounce it as a method, 
that we create a school for the new social order out of whole cloth, that 
creation rather than scientific achievement is now the order of the day. 

The open discussion of these views in the educational press is stimu- 
lating and obviously useful, The changes in practice of some of our 
leading graduate schools of education in reducing the amount of em- 
phasis upon scientific work is to be viewed with concern. 

If the offering of a non-research degree, such as the Doctor of Edu- 
ration, results in shifting the emphasis to practice rather than research, 
the number of people who are competent to contribute to the scientific 
knowledge in this field will be notably decreased. In view of the ad- 
vances in educational practices which even a short thirty years of sci- 
entific research have made possible, it seems unthinkable that educa- 
tional leaders will now turn back to speculation and authority. Yet, 
with science gone, what else remains but these? 

The time for scientific contributions to education was never so op- 
portune as now. The age of easy expansion in education is over, We 
have witnessed in successive waves popular interest in achievement 
tests, mental tests, new-type examinations, ability grouping, curriculum 
revision, project and activity movements, and administrative schemes 
without number. Popular exploitation of these is past. Real scientific 
achievement is now possible. 


A statement concerning graduate work as continued pro- 
fessional preparation for teachers implies less emphasis on 


26 G. T. Buswell, “After Science—What?” Phi Delta Kappan, XVII (February, 1935), 
214-5. 
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production of research than is advocated in the preceding quo- 
tation, but with some stress on use of research findings.** 


Graduate work for teachers may be looked at from two angles. The 
first considers graduate study as a continuation of the period of pro- 
fessional preparation during which the teacher may supplement his 
previous preparation. The second considers graduate work as a dis- 
tinctly different element in the teacher’s preparation emphasizing 
research about education rather than the acquisition of additional 
teaching skill or teaching materials. It appears obvious from the list 
of suggestions just reported from those in charge of graduate and 
undergraduate units in education that the second of these points of 
view—the research side of graduate work—predominated. There is 
little doubt concerning the desirability of having a public-school 
teacher able to conduct research studies in his special field of teaching 
if the time spent in preparing him to conduct such studies is not out of 
all proportion to the value received. In other words the teacher’s pri- 
mary responsibility is to teach and if training in research produces a 
greater improvement in his teaching than would the same amount of 
time spent in other phases of his professional preparation or in obtain- 
ing additional knowledge of the subjects he teaches the research empha- 
sis in graduate work for teachers is justified. Public-school teachers 
should be open-minded about their work and ready to accept or try out 
new methods of teaching or new subject-matter content when such 
methods or content arrangement have resulted from researches in those 
fields. It is not necessary that a teacher be a trained research worker in 
order to have this attitude toward educational experimentation. It is 
conceivable that too much interest in educational research may even 
interfere with the efficiency of the teacher's classroom worok, 


Those who are responsible for training research workers in 
education will find an analysis of deficiencies in such training 
(Table XV) and of needed types of training (Table XVI) 
suggestive.” Only categories are listed which were mentioned 
by five or more of the investigators canvassed. The limitations 
of such an analysis are in its subjectivity. 

Before taking up a more detailed discussion of the require- 
ments for the Master’s degree and the Doctor’s degree, a brief 

ar. S. Evenden, Summary and Taterpretation, pp. 138:9) National Survey of the 


Education of Teachers, Vol. VI. Office of Education Bulletin, No. 10, 1933- Washington: 


Office of Education, 1935- , 
28 R. W. Tyler, “Training Courses for Research Workers,” Educational Research Bul- 


letin, XI (March 30, 1932), 160-79. 
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TABLE XV 
‘Types or DEFICIENCY IN TRAINING FoR RESEARCH 
Number 
Reporting 
Lack of an integral philosophy of education to which each research 


problem! is related ./5.c,0.:t. sys dateteecersccsiscerecersoue 29 
Lack of knowledge of actual conditions which define research 


problems strasa ania sew! na ea» 37a tetas Sere 22 
Lack of training in the use and interpretation of statistics. 17 
Lack of wide acquaintance with techniques for collecting iomas 

tion which are adapted to particular problems..............+ 16 
Too few types of research problems attacked while in training... 13 
Inadequate training in reporting research. 13 
Lack of training in interpreting findings. ... 12 
Too little field work 10 
Too little training in discovering significant problems... 9 
Lack of development of social personality... 8 
Too much course work...........ssee+seees 7 
Lack of training in planning research........... 6 
Too little supervision of students of research... 5 

TABLE XVI 
Types or TRAINING NEEDED FOR RESEARCH 
Number 
Reportiny 
Training in the use and interpretation of statistics 43 


The development of an integrated philosophy of education. . Hy ge! 
Training in reporting research............sseeseeeeeeceseees Bere oh 20. 
Broad training in fundamental fields of scholarship as well as in 

. education .. 
Practice in attacking a variety of types of educational DOE 

in real situations............. ara p Siate widhes whe a AT 14 
Participation in field work..... 
Practice in recognizing problems...... nee 
Teaching or administrative experience.................0..0.0005 10 
Training in techniques for collecting data which are adapted to 

particular problems .. 
Training in contro!led experimentation... 
Training in the construction and use of tests. 
Practice in interpreting research findings in a variety of fields... 
Apprenticeship in research..... AE Pe are ote Rage eat EPON 
Training in planning research....... pace E ere tees,» ha. 
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review of the expansion and problems of graduate work in this 
country is in order.?° 


With the extraordinary expansion of such fields as agriculture, indus- 
try, commerce, and education, graduate study and research turned in 
the direction of applications of truth and knowledge to immediate 
problems rather than toward general principles. This meant the study 
of many local and regional problems and the development of research 
in American institutions rather than to attend German and French 
universities, Such factors explain in a large measure the decrease in 
emphasis on foreign languages as requirements for the Ph.D. degree 
and the establishment of professionalized Doctors’ degrees such as 
Ed.D., Sc.D., Eng.D., J.S.D., etc. 

There has been considerable overexpansion of graduate departments 
in American institutions, raising a question as to how many of the 
seventy-five or more universities and colleges which grant the Ph.D. 
degree should do so and to what extent work should be duplicated. 
Undoubtedly five or ten strong centers in a highly specialized field 
would be much better than -fifty weak departments covering this area. 
The Master’s degree is awarded by 160 schools. 

The discovery each year of some 3,000 Doctors’ problems and of 
12,000 Masters’ topics is a matter of no small significance, suggesting 
that the graduate student in many instances must do considerable 
searching before locating an acceptable thesis problem, 

The foregoing facts imply that a rather difficult problem of artic- 
ulation exists between the graduate institutions and some 1,000 under- 
graduate colleges. The choice of a graduate school requires real thought 
and guidance, for which some assistance is available in the classified 
lists of higher institutions prepared by the Association of American 
Universities, and in the report prepared... for the American Council 
on Education.®° : 


The Master’s degree. A general statement of requirements 
for the Master’s degree, without special application to educa- 
tion, follows: ** 


There has been little change in the formal standards set up for the 
Master’s degree since 1900. 

The quantitative standards and requirements for the M.A. and M.S. 
degrees are practically identical. 


29W. C. John, Graduate Study in Universities and Colleges in the United States. 
Office of Education Bulletin, No. 20, 1934- Washington: Office of Education, 1935. Pp. 
xiv + 234. 

30R. M. Hughes, “Report of the Committee on Graduate Instruction,” Educational 
Record, XV (April, 1934), 192-234. 

31W. C. John, op. cit., pp. 151-2, 
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With the exception of three institutions that have endeavored to 
bring the Master’s degree to a level two years above the Bachelor’s 
degree, there is everywhere a general agreement of a minimum require- 
ment of one scholastic year’s work above the baccalaureate. The 
quantitative requirement of thirty semester-hours, including a thesis, 
or twenty-four semester-hours plus the thesis is a conventional prac- 
tice. 

In general, a foreign language requirement involving the knowledge 
of at least one foreign tongue is essential, but a few institutions or 
departments in certain institutions may waive this requirement. 

The thesis is still required in most schools. Only a few exceptions to 
this requirement have been observed. There appears to be little or 
no uniformity in the matter of the requirement of the final examination 
or in the method of conducting it. 

The Master’s degree has come to be highly differentiated, as far as 
designation is concerned. 


In 1930, there were 139 institutions that offered graduate 
degrees in education.*? Of these twelve offered the A.M. in edu- 
cation; seventeen, the M.S. in education; and twenty-four, the 
Master of Education. At least ror of the above-mentioned in- 
stitutions also offered the A.M. degree and fifty-six the M.S. 
degree. In most of these cases a major in education was 
permitted. Data giving the distribution of these institutions, 
with the Master’s as well as the Doctor’s degree offered, can 
be found in the Survey of the Education of Teachers.** The 
number of Masters’ degrees in education, involving a thesis, 
granted in 1933 was 3229 and in 1934, 2763. 

Detailed information is available in another source concern- 
ing the administration and requirements of the Master’s degree 
in education in sixty-four institutions: name of the degree, 
administrative unit offering the degree, prerequisites to can- 
didacy, differentiation of graduate from undergraduate work, 
credits accepted by transfer, advisement of students, adminis- 
tration of the thesis (fifty-one schools require it, eight permit 
substitutes, and five do not require it), final examination 

82 Ibid., pp. 149-50. 

83 W. C. John, “Graduate Work in the Education of Teachers,” Teacher Education 


Curricula, National Survey of the Education of Teachers, Vol. III, Part VI. Office of 
Education Bulletin, No. 10, 1933. Washington: Office of Education, 1935. 
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(forty-one schools require it and thirteen do not), major and 
minor requirements, courses other than majors and minors, 
total credits required, requirements of specified courses such as 
statistics (thirty-one schools) and research methods (twenty- 
five schools), course credit for the thesis, foreign languages 
(not typically required), and opinions concerning the purpose 
of the degree (a period of advanced study in fifty-two schools, 
and a research degree in twenty schools) .** 

The fact that a number of graduate schools have made of 
the Master’s program merely another year of training for 
teaching beyond the Bachelor’s degree and have eliminated the 
thesis as a requirement is not in keeping with the emphasis on 
training for problem solving, as expressed frequently in this 
book. It is vigorously maintained in a personnel study of 242 
students who received the Master’s degree in education at the 
University of Chicago that the major responsibility of the 
graduate school is to give a competent group of workers 
the scientific training which will result in critical productive 


scholarship." 


Numerous problems are suggested by the facts which have been 
presented. However, in the writer’s judgment, the major problem re- 
vealed is that of the selection of graduate students. A redefinition of 
graduate work is being forced. The efforts of the graduate school 
must be focused either upon the task of stimulating fundamental 
research on educational problems or upon the task of providing an 
additional period of training beyond the bachelor’s degree, but with 
emphasis upon practice rather than research. If the former purpose 
is adopted, the selection of students will need to be much more 
rigid than if the latter purpose is accepted. Both purposes are worthy; 
but the mixing of the two in the same department and the granting 
of the same degree for each will become increasingly difficult as the 
demand for the Master’s degree continues to increase. The Master's 
degree needs definition. Either it stands for preliminary training in 
research or it does not. Perhaps it may be desirable to consider it a 


84J. O. Powers, “The Administration and Requirements of the Master’s Degree in 
Education,” Practices of American Universities in Granting Higher Degrees in Educa- 
tion, Chapter II. Yearbook Number XIX of the National Society of College Teachers 
of Education. Chicago: University of Chicago Press, 1931. 

35G. T. Buswell, “A Personnel Study of Students Who Have Received the Master's 
Degree from the University of Chicago.” School and Society, XXV (June 18, 1927). 
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teaching degree. The writer does not so believe. The same type of 
outside pressure which now makes the Master’s degree in great 
demand will in another generation or sooner be applied to the Doctor’s 
degree. The problem is fundamental. Education must have men for 
scientific research; it must also have men skilled in teaching and 
administration, But the degrees which have formerly represented scien- 
tific achievement are more and more being desired by a selection of 
students whose chief interest is practice rather than research. The 
writer believes that a clear-cut differentiation between students inter- 
ested in research and students interested in practice is essential in a 
subject as new as education, Whatever may be the decision of the 
graduate school regarding its obligation to give training in practice 
and regarding its recognition of such training by a degree, its principal 
obligation will be the selection of a competent group of students 
and the giving to this group a rigid scientific training which will 
cumulate in critical, productive scholarship. This type of interest is 
utterly foreign to many first-year graduate students as now selected. 


Although this type of scholarship is too much to expect of 
the majority of first-year graduate students, the Master’s 
thesis has indirect values of importance as one measure of the 
level of scholarship attained, even though the contribution 
made to knowledge is only a minor one.*® 


The administrators and supervisors of our public schools show more 
interest in research than ever before. Fifteen years ago the educational 
conference which based its program on the findings of research at- 
tracted the attention only of specialized research workers, To-day at 
the meetings of the National Education Association the sections deal- 
ing with scientific investigation attract the largest audiences. Again, a 
decade ago professional magazines which reported research studies had 
a very limited subscription list. To-day these same magazines are 

found on the desks of high-school principals, superintendents, and 
other practical workers in the field. 

What has brought about this change of attitude? Could it be that 
these schoolmen have had sufficient experience in writing the Master’s 
thesis to learn respect for research and the research worker? Could it 
be that they started with one point of view and after collecting data 
throughout the year came to a realization that their previous judgment 
had been wrong? If it has done nothing more than to cause our 
administrators to collect all the facts before rendering decisions, the 
training is thoroughly worth while and should be continued. 


36“The Master's Thesis,” University of Pittsburgh School of Education Journal, ITI 
(May-June, 1928), 100-1. 
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One institution which required no Master’s thesis over a 
period of years has since 1929 changed the former policy, in 
view of the values expected to accrue to the student in the 
preparation of a thesis.’ 


The practice is expected to give the student some training in the 
application of scientific method; to enhance the student’s knowledge 
and skill in reading scientific studies through this application of 
principles of scientific techniques in the preparation of one such study; 
to develop the understanding and appreciation of the concept of 
scientific method through the preparation of a study in which the 
principles of scientific method are involved; to give practice in sensing, 
defining, and solving problems through the employment of scientific 
method; to give the student some training in using the principles and 
techniques of research in the hope that he may attack his future 
problems scientifically and engage to some extent in creative research 
work in the field of his special interest. It should be noted that the 
major emphasis in these statements of objectives is on training in 
the application of appropriate methods of research rather than in the 
intrinsic value of the contribution made. 


The Doctor’s degree. Until recently the Doctor’s degree 
had not changed its requirements for three quarters of a cen- 
tury ; even now few changes have been made outside of profes- 
sional fields such as education, despite the great educational 
and cultural changes of the intervening years. A brief introduc- 
tory statement of the standards and requirements for the Ph.D. 
degree in general follows: ° 


There has been little change since 1900 in the formal standards set 
up for the Ph.D. degree. 

The standards and requirements for the Ph.D. degree are being 
modified in the direction of flexibility. Qualitative standards are tak- 
ing the place of mere quantitative requirements. 

The requirements for the Ph.D. degree show considerable uniformity 
with respect to such general items as admission to candidacy and 
minimum residence requirements. That is, the regulations for admis- 
sion to candidacy provide in nearly all cases an adequate period for 
testing the candidate’s ability as a graduate student, and also providing 
a period in which an acceptable program involving a primary and 


87 Clifford Woody, “The Master's Thesis,” University of Michigan School of Education 
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secondary interest may be worked out. The thesis or dissertation is 
required in all cases, but there is considerable difference of view as 
to what constitutes a thesis which adequately measures or tests the 
originality and scholarship of the candidate and the value of the 
contribution. 

The publication of the Doctor’s dissertation in some form is 
everywhere required, with one exception. Only one institution requires 
full publication before the degree can be granted. But the require- 
ments for the publication of the thesis in many universities are so 
full of alternatives as to discourage a serious attempt at full publica- 
tion. In a considerable number of schools the method of publishing a 
university series of thesis abstracts seems to be an interesting com- 
promise. This may help the finances of the student, but it also reduces 
the significance of full responsibility on the part of the student in 
completely fulfilling his principal objective in the graduate school. 

While not altogether uniform as to procedure, the final oral exami- 
nation or defense of the thesis continues to be the universal practice. 
This is usually preceded by a comprehensive written examination. 

The creation of new types of Doctor’s degrees similar in general 
character to the Ph.D. degree has become increasingly popular. These 
new degrees are largely controlled by professional considerations and 
in a number of instances require of the candidate a period of practical 
professional or field experience. The emphasis is placed on the appli- 
cations of knowledge rather than upon the discovery of new knowledge 
or truth. These degrees are now fully recognized by a large group of 
leading universities and are on the whole considered to be the equiv- 
alent of a Ph.D. degree. 


The comparatively recent Doctor of Education degree is 
found to vary in certain respects from thè Ph.D. degree.*® 


The degree of Doctor of Education is offered by twenty-one univer- 
sities and colleges, or in one third of all the colleges and universities, 
excluding teachers colleges, that offer the Ph.D. degree with specializa- 
tion in education. The Ed.D. degree is not offered as the only Doc- 
tor’s degree in the twenty-one institutions mentioned, but it is offered 
parallel with the Ph.D. which is also offered in the same institutions. 

The characteristics of the Ed.D. degree may be summarized as 
follows: 

A It gives major emphasis to the constructive solution of difficult 
practical problems by well-proven methods or techniques. 

2. It gives great emphasis to the mastery of educational subject- 
matter in specialized fields rather than in prosecuting original research. 

The emphasis of the Ph.D. with a major in education tends in the 


39 Ibid.. pp. 189-00, 
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direction of historical or scientific studies of education, including the 
development of new theories and new techniques. 

As yet there is no complete agreement of authorities as to the 
different functions of the Ed.D. and Ph.D. degrees, as we find many 
cases in which the Ph.D. includes all of the functions indicated for 
both degrees. 

From the standpoint of administration the control of these degrees 
tends to be almost entirely separated. The Ph.D. with major in 
education comes under the control of the graduate school of arts and 
sciences or equivalent, and the Ed.D. comes under the control of the 
school of education. 


A major objective of the Ed.D. degree seems to be the ad- 
vanced training of administrators and supervisors, and the 
preparation of master-teachers. Variations from the Ph.D. 
requirements involve less emphasis on the foreign-language re- 
quirement, teaching experience, special knowledge of the tech- 
niques of educational research, and in certain cases preliminary 
demonstration of ability to undertake graduate work. 

According to Raup, a working distinction between Doctor 
of Philosophy research and Doctor of Education professional 
attainment, stressing the functions of the latter degree and 
training program, is as follows: *° 


1. Increased understanding and improved conceptions of the delib- 
erative act itself 

2. Reconsideration of general principles of conduct and policy, 
especially as these apply more directly to the professional field 

3. Advancement of the candidate’s and the profession’s under- 


standing of fact, theory, and law 
4. Insight into the relation of systematic reasoning to the success 


of the educator’s everyday decisions 
5. Provision for reincorporation of the foregoing newly considered 
generalizations into the process of deliberation 


In tools of research, especially foreign languages, revision of 
requirements has been slow. Bay, as a result of an analysis of 
204 theses in the series, Teachers College Contributions to 
Education, Columbia University, raises a question concerning 


40R, B. Raup, “Division of Labor in the Study of Education,” Teachers College 
Record, XXXVII (November. 1935), 119-29. 
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the desirability of the foreign-language requirement for the 
Doctor’s degree in education. Stated in terms of percentages 
employing a given tool, the results of his analysis are: 1905- 
1915, foreign languages 38, statistics 41, neither foreign lan- 
guages nor statistics 30; 1916-1926, foreign languages 16, 
statistics 78, neither foreign languages nor statistics 18.“ This 
trend probably has continued during the past decade. 

Betts insists that a number of factors be considered in a 
given field before deciding upon the merits of a foreign-lan- 
guage requirement for the area involved.” 


The foreign language question for Ph.D.’s is not settled—it is just 
opening up. The report of the present inquiry already referred to 
contains much of both fact and opinion. A further study is still 
needed, Let each separate field of scholarship appoint a fact-finding 
commission to discover: (1) the amount of significant material ap- 
pearing over a given period in each foreign language; (2) the propor- 
tion of this which is translated into English within a given time; (3) 
the amount and adequacy of the abstracting done; (4) the practica- 
bility of a system of departmental translators; (5) workable plans 
for making more practical use of foreign language materials in courses, 
seminars, and researches in the graduate school. 

Then let some agency bring these fact-studies all together and give 
us their results. With this done we should be nearer the solution of 
the language question than we are at present. 


The oral examination, which is commonly required of 
Masters’ and Doctors’ candidates, has been subjected to experi- 
mental evaluation with a real need for improvement indicated 
in terms of objectivity, reliability, and validity.* Another ux 
vestigator finds the oral examination not “so objectionable” as 
a measuring device when definitely planned, carefully prepared 


41 James C. Bay, “Tools of Research Needed in the Preparation of Dissertations for 
the Doctorate in Education,” School and Society, XXV (January 8, 1927), 53-4- 

42 G. H. Betts, “Language Requirements for Doctors of Philosophy,” School and 
Society, XXXI (March 8, 1930), 343-6. 

, and R. A. Kent, Foreign Language Equipment of 2325 Doctors of Philosophy. 
Northwestern University Contributions to Education, No. 2. Bloomington, Ill.: Public 
School Publishing Co., 1929. Pp. 152. 

43 Elinor J. Barnes and S. L. Pressey, “The Reliability and Validity of Oral Examina- 
tions,” School and Society, XXX (November 23, 1929), 719-22. 


S. L. Pressey, L. C. Pressey, and Elinor J. Barnes, “The Final Ordeal,” Journal of 
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for (by all concerned), and systematically administered.** 
Pressure of time is forcing more written Masters’ examinations. 

An analysis of limitations affecting the candidate when he 
is working on his Doctor’s dissertation includes: (1) the short 
period of time available for graduate study, (2) the cost of 
carrying on a comprehensive piece of research, (3) the limited 
professional training of the student at the time he must select 
the subject for his dissertation, and (4) the lack of mastery of 
the techniques which the student must use in his research. 
Suggestions for meeting these difficulties are: (1) earlier selec- 
tion of the problem, (2) more careful supervision of the 
selection of the problem by the faculty, (3) greater coördina- 
tion of research projects, (4) devotion of more time in depart- 
ments of education to informal but critical courses dealing with 
the techniques of research at the time candidates are making 
their selection of problems, and (5) an attempt by institutions 
to provide subsidies for important and desirable types of re- 
search which are too expensive to be undertaken by an indi- 
vidual student.“ 

Symonds criticizes graduate studies, with special applica- 
tions to Doctors’ dissertations, in terms of six common short- 
comings: ** (1) that the emphasis on the gathering of facts 
has been poorly correlated with the application of these facts 
to the educational program, (2) that the background of infor- 
mation and understanding of the student is often meager and 
shallow, (3) that certain fundamental assumptions which are 
often tacitly understood give the research a particular bias 
and make it of limited value, (4) that theses sponsored by and 
often financed by groups or theses addressed to certain groups 
are often biased, (5) that choice of methods of investigation 


440, C. Trimble, “The Oral Examination: Its Validity and Reliability,” School and 
Society, XXXIX (April 28, 1934), 550-2- oat k 
, The Final Oral Examination: Its Limitations and Its Possible Improve- 
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or of the instruments to be used in the investigation is often 
faulty, and (6) that there are a number of technical faults 
which are commonly seen in the statistical methods used and 
in the preparation of the manuscript. 

Certain criticisms of educational research, with special ap- 
plication to theses, as conducted in some institutions, are cited 
as follows: *7 


1. Graduate students work on theses in which they are not inter- 
ested and do not comprehend in order to satisfy the thesis requirement 
for the doctorate and their adviser’s personal interest in a subject 
which is a part of his major work. 

2. Studies are begun, but not completed, as evidenced by phrases 
introducing conclusions, such as, “It would seem,” “It appears,” “As 
far as we were able to proceed, we may conclude,” etc. 

3. Investigators are shying away from significant problems on the 
grounds of their prohibitive comprehensiveness, although teachers must 
meet and solve these problems. 

4. Students lack broad and thorough training in research technique. 

5. Studies are too hurried. 

6. Students fail to classify their research along the lines of the 
major divisions of research. 


A list of criteria for educational dissertations, based on char- 
acteristics common to twenty-five good dissertations in- 
cludes: 43 


Creativeness 

Wise choice of topic 

Evidence of ability 

Fundamental assumptions considered and justified 
Appropriateness of procedures to the topic 
Quality and quantity of data 

Sound analysis of data 

Interpretation of data 

Presentation 


COSTE aaea ES PA aE 


o 


Another suggested list of standards for the dissertation in 
education includes these essential characteristics : 4° 


47 T. W. H. Trion, “Criticism of Present-day Educational Research,” North Central 
Association Quarterly, I (December, 1926), 390-3. 

43 P. M. Symonds, “Characteristics of a Good Dissertation in Education,” Teachers 
College Record, XXXIV (January, 1933), 312-26. 

49 L. B. Rogers, “Desirable Standards for the Doctor’s Dissertation: Should the Dis- 
sertation be Based on Other Forms of Productive Work in Addition to Research?” 
Educational Administration and Supervision, XVI (December, 1930), 683-92. 
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1. Based on a discernment of and responsiveness to social needs 
2. Presents a clear, concise formulation of a problem or working 
hypothesis 

3. Reveals a definite placement of the problem 

4. Makes evident the relevancy and adequacy of the materials and 
techniques used 

5. Observes continuity and unity in organization of materials 

6. Uses the most effective means available for a clear, forceful 
presentation 

7. Gives a complete and consistent interpretation of the data 
8. Indicates the bearing of the results on accepted principles and 
practices, and the need for further research 


Stated in another way, three fundamental criteria for the 
Doctor’s dissertation in education have been suggested, which 
represent fairly well the departmental policy of one of the 
major graduate institutions : °° 


First, the basic requirement of any research is a body of objective 
data. Speculation, a priori theorizing, or hypothetical statements are 
never substitutes for facts. Unless a problem is susceptible to factual 
treatment, it is not a suitable topic for research, There are such topics. 
Certainly, the possibility of scientific research on such a question as, 
“Does education through discipline produce greater individual satis- 
faction than education through freedom?” is doubtful. One may 
speculate and philosophize about this question, but it is difficult or 
impossible to find facts from which one may answer the question. It 
may be possible to do more than philosophize about the problem if 
one is willing to cease the exclusive use of opinions regarding satisfac- 
tion and to gather some objective facts from the observation of 
behavior of persons variously educated. This is not to be interpreted 
as a denial of the value of philosophical thinking, but rather as a 
vigorous objection to the substitution of such thinking for objective 
data, Furthermore, the process of dealing with a problem without 
first securing a body of facts is a different precess from that connoted 
by the term research. 

Second, objective facts are necessary, but a simple tabulation of 
facts does not constitute acceptable research at the doctoral level. 
Until some degree of generalization is reached, the project is unfin- 
ished. One might collect facts regarding any school practice and 
organize them into frequency tables, but until some general relation- 
ships among these facts are discovered the project is still incomplete. 
The nature of the generalizations may vary from field to field, but 

50G. T. Buswell, “The Doctor’s Dissertation,” Journal of Higher Education, IIL 
(March, 1932), 139-46. 
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regardless of whether the problem be in the history of education, in 
administration, in methods, or in psychology, until some degree of 
generalization is reached, the investigation has not passed beyond the 
bounds of clerical industry. Needless to say, much of this industrious 
routine work may be useful, but it is not research. The essential 
characteristics of the research worker are the intellectual insight 
necessary to see relationships which are not obvious to the lay observer 
and the ability to generalize concerning these relationships. 

Third, a routine application of scientific principles to a practical 
problem is not research. For example, in one of the letters the location 
of school plants as a thesis problem was suggested. This might or 
might not involve research. If the commonly known rules relating to 
population centers, dangerous highways or railroads, and the like, only 
were applied to the selection of school sites, no discovery of new 
truths and no advancement of general scientific knowledge would 
result. This problem might be attacked, however, in a way which in- 
volved a high type of research. Through an analytical study of all 
the factors affecting the location of school buildings some new factors 
that generally affect the location of school plants might be discovered; 
a new technique by which to obtain or to interpret the facts gathered 
might be developed. The objection to many so-called “practical prob- 
lems” is that they are merely applications of well-known procedures— 
the truth that is “discovered” is too often new only to the man who 
discovers it. 


A canvass of the practices of twenty-nine institutions which 
gave more than one Doctor’s degree during the decade from 
1918 to 1927 resulted in the following recommendations: °* 


1. A minimum of twelve to fifteen semester hours of undergraduate 
“work in education for admission to graduate study 

2. A written preliminary examination to be given toward the end 
of the second year of graduate study 

3. A final examination, at least partially written, and general in 
Scope rather than confined to the thesis or to the thesis and related 
matters 

4. Conservative restriction of the amount of work carried by as- 
sistants, instructors, and other students who have either part-time or 
full-time employment 

5. The requirement of one or two carefully planned sequences of 
courses 

6. The requirement of a few basic courses, such as educational 


51 W. S. Monroe, “A Survey of the Requirements for the Doctor of Philosoph: in 
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philosophy or theory, educational statistics, thesis-writing, and educa- 
tional research 

7. Supervision of thesis work by a committee, probably a commit- 
tee of three, rather than by a single member of the staff (This super- 
vision should begin with the selection of the thesis problem.) 


Studies of supply and demand in the graduate field indicate 
that too great specialization is undesirable on the part of the 
Doctor’s candidate, since he may have to carry on his future 
work in some area other than university teaching and re- 
search.°? Business, politics, social service, and secondary 
schools have been suggested as future fields of activity for tne 
Doctor of Philosophy in this country and in Europe." It has 
been recommended that psychologists be trained to provide 
professional psychological service in the fields of business per- 
sonnel, student personnel, industrial psychology, vocational 
guidance, and child welfare." 

Other surveys of requirements for the Doctor’s degree in 
education * and in psychology ** are available. Of course, the 
wise prospective graduate student will examine in the cata- 
logues the specific requirements for degrees of the institutions 
in which he is interested. Some evidence concerning the rating 
or standing of graduate departments of education is available, 
although the method used (opinions of a committee of fifty- 
four “scholars” in the field represented) has met some criti- 
cism.*7 Sixty-one institutions conferred 1300 doctorates in 
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education between 1928 and 1932; of this number thirty 
schools were rated as adequately prepared to do so, and only 
ten were considered “distinguished.” The number of Doctors’ 
degrees conferred in education during 1933 was 370; in 1934 
the number was 46ọ. 

Tt should not be assumed that the research worker’s training 
is finished on completion of the doctorate program. An increas- 
ing amount of post-doctorate training is being encouraged by 
higher institutions, research organizations, and fellowships.** 

Problems involved in graduate training programs are dis- 
cussed in numerous other references in the chapter bibliog- 
raphy, a part of which appear under the names of: Alexander, 
Almack, Ashbaugh, Barr, Bay, Betts, Briggs, Brown, 
Buchanan, Buckner, Buswell, Cooke, Crawford, Dale, Eaton, 
Edwards, Fenton, Fernberger, Ford, Freeman, Good, Greene, 
Harris, Heller, Higbie, Hill, Hills, Hunt, Jones, Judd, Kunkel, 
Lowell, McCall and Ruger, Mathews, Mead, Nelson, New- 
comb and Watson, Palfrey, Pechstein, Pressey, Robertson, 
Rogers, Ruediger, Russell, Seashore, Smith, Symonds, Tufts, 
Walker, Watson, Williams, and Wykoff. 

Coérdination and coöperation in research. The condition 
deplored in 1927 by an editorial writer, who urged that some 
central coérdinating agency make known the various investiga- 
tions completed and under way, in order to avoid competitive 
research and duplication of effort, has been partly corrected by 
the guides enumerated in Chapter IIT; however, the problem 
is not yet completely solved.” The movement toward integra- 
tion of the efforts of educational investigators is significant for 
all who participate in, direct, or apply the results of research. 
Illustrations of the results of coérdinated investigational effort 
are found in Chapter III in the form of nation-wide surveys 
of various fields, the year-books and publications of great 
foundations and professional groups, comprehensive summa- 
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ries of research, and the work of research bureaus and or- 
ganizations. 

Three requirements for codperative investigation have been 
named: (1) a well-selected committee to define major issues 
in the field of investigation, to suggest methods of investiga- 
tion, and to review findings previous to publication; (2) a 
personnel to conduct the investigation sponsored by the com- 
mittee; and (3) some provision for travel expense.® 


Provided with these essentials, group research bears fruit. It avoids 
some of the weaknesses which frequently characterize individual effort. 
The topics of investigation are likely to be well chosen, and bias in the 
interpretation of results is reduced. The best methods of investigation 
are likely to be employed. Most important of all, a group of people 
with prestige are familiar with the findings of the study which, there- 
fore, is far more likely to get over into practice. It is at this point 
that many individual investigations fail. Every time one prepares a 
comprehensive bibliography he is struck by the number of significant 
researches already in existence which have received little or no atten- 
tion as far as modified school practice is concerned. 


Four types of codperating groups have been proposed: ° 
(1) intra-institutional groups in which tasks are assigned by 
administrative officials with little or no consideration for the 
wishes of individuals; (2) intra-institutional groups in which 
tasks are chosen by the group concerned in accordance with 
the principle of the majority rule; (3) inter-institutional 
groups in which the tasks are assigned by administrative offi- 
cials, committees, etc.; and (4) inter-institutional research 
conducted by self-initiated groups of individuals of like in- 


terests. :) 


Doubtless each of these methods of securing group activity 
has its own peculiar values, depending upon the nature of the 
problem, and the competency and the attitude of the indi- 
viduals concerned, but the writers should like to emphasize the 
merits of the last. The real advantages of this type of codpera- 
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tion are: (1) It arises out of each individual’s own felt needs, 
difficulties, and interests; the advantage of having individuals 
work at research that they think important is tremendous. 
(2) It is for the most part independent of financial considera- 
tions. Far too many persons appear to have the idea that re- 
search depends upon money. Although money is always a 
convenience, the essential element in any piece of research is 
the thinking, and when money is employed to remove an 
otherwise intelligent and competent research worker from im- 
mediate contact with his data, the stimulation and the con- 
stant checking and rechecking afforded by these data are 
removed. In the main, the extensive use of the so-called 
statistical clerk is based upon a misconception of the nature 
of research. And (3) this type of coöperation is independent of 
committee appointments and political preferences. Out of the 
confusion between political and intellectual leadership many 
persons otherwise competent to do research are heard to at- 
tribute their inactivity to failure to secure official preferment. 
Again, the authors would like to say that, aside from the per- 
sonal gratification which always arises from these official 
preferences, they in no way enhance the individual’s ability to 
do research. The type of coöperation urged is free and un- 
limited except by the capacity, initiative, and leadership of 
those who undertake it. 

Specific suggestions concerning how graduate theses may in- 
volve a unified pattern or be parts of a larger, planned program 
of research end as follows: ¢ 

Here, then, is the line of development which the administration of 
educational research in America will probably find it wise to follow. 
It will diligently avoid any semblance of imposition of a topic from 
above as well as the tendency to follow too literally the personal 
preferences of the individual student by merely regulating his fancies 
when they become too unrealistic. In place of these obsolescent 
devices, it will move to substitute a system of group or codperative 


planning of an extended research program wherein the work of each 
member falls into its prescribed niche. Continuity rather than discrete- 
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ness will be its watchword. So far our educational activities have been 
hit-or-miss shotgun forays into the territory of the unknown; hence- 
forth, we must have mass invasions moving toward the deliberately 
arranged conquest of key positions. 


Another worker criticizes the poor contacts which thesis ad- 
visers experience with the schools studied in educational 
research and the resultant ill effects on the theses and educa- 
tional progress: He submits a plan of research organization 
which unites the skill of the technical investigator and the 
need for more definite.knowledge felt by teachers. A new offi- 
cer, the Coördinator of Research, is to be placed in the school 
system, with these duties: °° 

rt. To make himself known in the area 

2. To familiarize himself with research needs in the area 


3. To assay the caliber of graduate students doing research in the 


institution with which he is connected 
4. To bring into contact the research opportunities and the properly 


qualified students 

The preceding discussion has stressed the merits of co- 
operative or codrdinated investigation. It is not to be assumed 
from such statements that there is no longer any place for the 
individual or pioneer type of research, but rather, that a proper 
balance between individual initiative and large-scale planning 
should be established in research. Two extremes—at least, 
widely divergent schemes—in educational investigation are 
represented in the largely individualistic research of America 
and in the current codperative plan of Australia. In Australia, 
where educational research is rather new, a planned program 
has been proposed which will coérdinate the work of the vari- 
ous individuals and direct it toward goals set up by the edu- 
cational authorities and leading workers. However, the 
codrdinating agency, the Australian Council for Educational 
Research, does not seek to centralize completely or control 
research, but rather to articulate the efforts of individual in- 
vestigators and of committees covering problems on a state- 
esearch,” Progressive Educa- 
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wide basis. The work of the Council, such as testing, is carried 
out in the schools on a codperative rather than a compulsory 
basis. 


Both the American and Australian plans have their advantages as 
well as disadvantages. One need but survey the wide variety of re- 
search that has been undertaken in this country to appreciate that 
the independent worker has made large contributions to the body of 
available scientific information dealing with educational problems. In 
many fields there have been highly important findings that have been 
incorporated into our educational practices. The independent worker 
has felt free to explore leads of interest to himself. It must be ad- 
mitted that some of these studies have not appeared to be immediately 
pertinent; however, one must admit that this may be because of our 
inability to see the possible relationships rather than the lack of value 
of the findings. 

A combination of the American and Australian plans seems highly 
desirable. On the one hand, one would want to do nothing to crush 
the fine spirit of independent individual research which has resulted 
in the development of an ever-growing body of educational truth. 
On the other hand, there is every reason to believe that a planned 
program of codperative research dealing with crucial issues in present- 
day education would be most helpful. This end might well be accom- 
plished through the coérdinated action of the various scientific bodies, 
both state and national, dealing with aspects of educational problems, 
The accomplishment of such an end demands real educational states- 
manship. Unless this is done, the alternative may be the establish- 
ment of some central authority which might arbitrarily take upon 
itself the setting up of standards and criteria for evaluation. 


* When the educational history of the first half of the twen- 
tieth century is written, the third decade of this period will 
be credited with the initiation of a program of codrdination 
or coöperation in educational research. This type of coördi- 
nated investigation and publication should break down ex- 
cessive isolation in research, prevent wasteful and unnecessary 
duplication of effort, and distribute the results of major sci- 
entific studies to field workers. It seems entirely reasonable 
to expect within the next decade in each of the subject-matter 
fields a survey similar to the Modern Foreign Language Study 
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and a summary of the available factual studies similar in com- 
prehensiveness to that of Gray in reading.” Whatever the 
future may hold, however, in the development of codrdinated, 
large-scale research, there is encouragement for the present 
in the progress reported. The obligation of the field worker is 
to keep informed of important studies in progress, to codper- 
ate wherever possible, and to use the findings reported in the 
improvement of teaching, curricular, or administrative pro- 
cedures. 

Bureaus and departments of research. Bureaus and depart- 
ments of research in city school systems, state departments of 
education, state educational associations, and higher institu- 
tions constitute important agencies in the supervision of re- 
search, and in the codrdination of the investigational effort of 
both field worker and graduate student. An excellent summary 
statement of the organization, functions, and investigational 
work of research bureaus follows: ° 


Growth in number of bureaus. It has been shown that the movement 
for organized research in education is comparatively recent, for it 
was not until the survey and testing’movement was under way that 
city school systems and State universities began the organization of 
bureaus of research, From the establishment of the first bureaus 
about ror2 the number has grown rapidly until there are at present 
about 150 bureaus in city school systems and about ninety-five others 
divided among State departments of education, State educational asso- 
ciations, State universities and colleges, teachers colleges, and secondary 
schools, About half of the bureaus have been established since 1925 
and about a fourth since 1928. 

The director of research. Exactly half of the full-time directors of 
research have ranks equivalent to that of assistant superintendents of 
schools. Part-time directors devote the remainder of their time to a 
diversity of other duties. The directors are usually responsible to the 
superintendent of schools and since many of them are in the line of 
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authority they are given the right both by authorization and implica- 
tion to investigate problems in any field of education. The directors 
have had on the average less educational experience than superintend- 
ents or principals, but about the same as other administrative staff 
members. However, some directors of research have not worked in 
types of school positions that would give them a basis of practica! 
school experience. Almost three fourths have never taught in city 
elementary schools; about three fifths have not been principals of 
elementary schools; and the same proportion have not been principals 
of secondary schools. On the other hand, directors of research, in 
terms of academic degrees held, are considerably better trained than 
other school officers. More than half of them hold Masters’ degrees, 
and more than a fourth hold Doctors’ degrees. In addition, most of 
them Have earned their advanced degrees in the field of education. At 
the time of inquiry the median salary paid full-time directors was 
about $4,000. 

The staf of research bureaus. The staff of the median research 
bureau consists of the director and three assistants; the helpers are 
about equally divided between professional and clerical grade. About 
half of those professional assistants in the bureaus, who devote only 
part time to research, are elementary-school teachers. Part-time clerical 
assistants are frequently high-school pupils. 

The budget of research bureaus. The median salary and operating 
budgets of research bureaus were about $9,700 and $2,000 respectively. 
In view of the fact that the'median bureau conducts twenty-one 
studies a year, it is difficult to see how comprehensive investigations 
can be made with the personnel and the resources available for research 
work. The financial resources must explain in part the limited types 
of researches reported. Classification and analysis of researches reveal 
that many of the studies involve merely the compilation and presenta- 
tion of facts, and that the staffs of many bureaus are absorbéd in 
duties and functions which are more of an administrative and super- 
visory type than of a truly research character. [Such work is useful, 
but is not the major function of a research bureau. ] 

Administrative and supervisory functions performed. As already in- 
dicated, large proportions of the directors of research perform duties 
and functions which are not of a research nature. No attempt is made 
in this report to show that these administrative functions are not 
important. However, analysis supports the belief that these functions 
are not always in the nature of true research nor inevitably functions 
which should be delegated to research bureaus. The compilation of 
data, the administration of achievement and intelligence tests, and the 
classification of school children are important functions, but the admin- 
istration of these activities is not research in the strict sense. At least 
school authorities should not confuse these activities with research and 
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assume that, having provided bureaus so engaged, substantial research 
contributions are being made. 

Studies which bureaus may make. Bureaus of research are authorized 
to make studies of pupil failures, promotions, and testing more fre- 
quently than of certain other problems such as employment service, 
clerical service, teaching methods, and supervision. The data did not 
permit an analysis of the reasons why some types of studies are au- 
thorized more frequently than are others, but size of the system seems 
to influence the scope of activities. 

In general, bureaus of research have actually made studies most fre- 
quently in the fields in which the greatest proportions of systems have 
been authorized to conduct investigations. Large numbers of studies 
have been made in the fields of testing, promotion, retardation, and 
elimination, and relatively few studies have been made in such fields 
as teaching methods, supervision, and library service. 

Studies made during 1929-30. Directors of research of fifty-three 
city bureaus reported a total of 1,116 studies in progress during the 
year 1929-30. The median bureau had in progress about twenty-one 
researches, As previously pointed out, the type of assistance fur- 
nished the director and the amounts which may be spent for the 
operating expenses of the bureau raise a presumption of doubt con- 
cerning the quality of many of these researches. Half of the studies 
in progress during the year were reported as relating primarily to 
the field of secondary education. Slightly more than half of the studies 
in progress during 1929-30 were later mimeographed, but relatively 
few have been either printed or published in educational periodicals. 
Only a few of the studies conducted in city bureaus have been sub- 
mitted as theses for advanced degrees and about ro per cent were 
reported by the directors of research as affecting, in any marked way, 
school organization and practice. The data do not permit an analysis 
of the extent and manner in which these studies have actually affected 
educational practices. This problem should be a fruitful field for fur- 
ther investigation. 

Codrdination of research activities. Research activities of city bu- 
reaus are codrdinated with those of other agencies inside or outside 
the system in only three fifths of the cases. The research bureau 
provides such coérdination as is attempted within city school systems 
by outlining the projects, by advising the staff members, and by 
exchanging the results of investigations with other bureaus or research 
agencies. A number of directors indicate that the bureaus attempt to 
develop a program of research activities and to guide and direct 
research work of ceachers within the system. 

State departments of education and State education associations 
have an opportunity of developing state-wide programs of educational 
research. Only a few States have attempted much along these lines 


750 EDUCATIONAL RESEARCH 


Research by persons not connected with research bureaus. Research 
has come to be considered by educational leaders as a function of 
teachers and principals, but to date few boards of education have 
made provisions enabling teachers to carry on research activities on 
school time. The number of researches carried on by staff members 
not connected with research bureaus is small and relatively few are 
ultimately mimeographed or printed. The real incentive for more than 
a third of the studies made by individuals unconnected with bureaus 
is the earning of advanced degrees. Research within individual second- 
ary schools is even more limited in extent. Large proportions of the 
high-school principals, both of the schools reported as outstanding in 
research and of those selected at random, stated that no research was 
being conducted in the schools. As a result, if the contention is that 
teachers and principals should carry on research, practice lags far 
behind theory. At present little research is being carried on in second- 
ary schools except by city bureaus of research, and in general even 
the investigations of these bureaus do not interpret the data and 
recommend improvements in school practice. 

Classification and analysis of researches made. In general, the re- 
ports of investigations submitted to the Survey indicate that bureaus 
of research in city school systems devote the major portion of their 
time and energies to the compilation of facts and statistics, However, 
this conclusion is not unexpected in view of the many administrative 
and supervisory functions delegated to these bureaus. They do not 
have the time or the resources to permit many investigations of a 
true research nature. It is admitted that facts and statistics concerning 
the school systems should be available to the superintendent and 
board at all times, but it is highly unfortunate that more bureaus do 
not conduct investigations of educational problems holding more 
promise of modifying and improving school administration, class-room 
techniques, ‘teaching methods, and other educational practices, If 
bureaus of research in city systems are to lead the way to a sounder 
educational program, they will find it necessary to place emphasis on 
fundamental problems of educational practice rather than on the mere 
compilation and publication of facts and statistics. 


ment in education, after all, is relatively new and each bureau and each 
school have had to work out their own problems and doubtless must 
at first justify their existence by rendering obvious services to admin- 
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substantial programs of research in these bureaus, and these, in turn, 
should stimulate larger proportions of individual members of school 
staffs not officially connected with the bureaus to carry on significant 
studies. Much assurance may be taken from the fact that at least a 
small number of bureaus in city systems of the country are prosecut- 
ing estimable investigative programs. The existence of a large number 
of bureaus of research which have made beginnings and of at least a 
small number of bureaus with strong investigative programs is a 
prophecy of generous spread of research activity outside higher institu- 
tions that will help to elevate education to unquestioned professional 
status. 


Four ways of promoting research among field workers have 
been stressed by research bureaus in state departments of edu- 
cation, especially New York State: ° 


t. Preparation of bibliographies of studies which pertain particularly 
to state educational conditions 

2, Gathering of occasional statements from educators concerning 
needed research in the state 

3. Assistance, through bibliographies and actual conducting of 
studies, to committees at work on such problems as tenure, salaries, 
curriculum, health education, radio education, correspondence courses, 
etc. 

4. Aid to individuals such as graduate students and field workers, by 
criticizing and planning studies 

From the point of view of the higher institution and its 
bureau of research, with respect to the administration of in- 
vestigational projects, three types of field research have been 
distinguished.” 


1. Individual research 
(a) The graduate thesis 
(b) The minor problem, worked out with the guidance of an 
adviser on the campus staff 
2. A codrdinated program of minor studies through guidance of the 
investigators, who select problems with some relation to a larger 
pattern or whole 
3. Major programs of research formulated by the institution which 
permits students to work on individual sections, for example, school 
68 Warren W. Coxe, “Functions of Research Bureaus in State Departments of Educa- 
tion in Promoting Research Among Field Workers,” Journal of Educational Research, 
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building construction costs and rates of obsolescence, and curriculum 
building 

Numerous other discussions of research bureaus and of 
foundations, scientific societies, and professional organizations 
which play a part in the direction and coérdination of research 
are represented in Chapter III and in the bibliography of the 
present chapter under the names of Bain, Chapman, Charters, 
Coxe, Cubberley, Gee, Herbst, Hunter, Ogg, Rankin, Sears, 
Spahr and Swenson, Withers, and Wright. 

Concluding statement. The need for productive scholarship 
and research ability as one of the highest types of educational 
leadership has been indicated, although recognizing the exist- 
ence of other important types of leadership in the areas of 
teaching, administration, interpretation, and innovation. It 
has been urged that to be most effective, the teacher, adminis- 
trator, or supervisor requires training in problem solving and 
investigational procedures, finding nothing incompatible in 
this proposal that both field service and research activity are 
important functions of the best prepared types of educational 
workers. The possible contribution of the graduate school to 
the training of such workers has been outlined. The movement 
toward codrdinated and codperative research has been re- 
viewed, together with the part played by research bureaus and 
organizations in the direction and supervision of educational 
investigation. 


PROBLEMS AND EXERCISES 


1. Select two recognized graduate departments of education and 
compare their requirements for the Master’s degree in terms of the 
principles stated in this chapter. It may be necessary to secure cata- 
logues from the schools selected, if such information is not in the 
library. 

2. Choose a bureau or department of educational research (uni- 
versity, state, city, or otherwise), and prepare a statement of its 
organization and activities current at the time this assignment is 
undertaken. The annual reports (where issued) of such research agen- 
cies and the Department of Research News of the Journal of Educa- 
tional Research may be consulted as starting points. 
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“Graduate Study and Research in Education,” High School 

Teacher, V (May, 1929), 148-51, 164. 

Goong, J. P. “The Doctor of Philosophy and His Business,” Science, 
LXIX (March 22, 1920), 318-21. Discusses the training of the Ph.D. 
and the subsequent research activities of the Ph.D. 

Goopman, N. G. “More about Ph.Ds,” School and Society, XXIV 
(October 2, 1926), 429-30. Advocates a greater interest in the world of 
human affairs on the part of graduate students and teachers. 

Greene, H. W. “Negro Holders of the Ph.D. Degree,” School and So~ 
ciety, XXXV (April 16, 1932), 542-4. 

—— “The Present Status of Negro Doctorates,” School and Society, 
XL (September 22, 1934), 388-9. 

Haccerty, M. E. “Improvement of College Instruction through Educa- 
tional Research,” Educational Record, XII (January, 1931), 43-60. 

* Haccerry, M. E. “Occupational Destination of Ph.D. Recipients,” Educa- 
tional Record, 1X (October, 1928), 209-18. Reports data from three 
public and four private universities concerning the professional 
work taken up by Doctors of Philosophy on leaving the university. 

—— “The Professional Training of College Teachers,” North Central 
Association Quarterly, II (June, 1927), 108-23. 

Hate, Genevieve. “Desirable Types of Advanced Training for Teachers of 
English in Secondary Schools,” Educational Administration and Super- 
vision, XXII (January, 1936), 61-71. 

Harris, R. G. “The Doctorate and the Depression,” School and Society, 
XXXVI (October 15, 1932), 498-500. 

Heiter, Otto. “The Assumption of Graduate Work by Undergraduate 
Institutions,” School and Society, XXXIX (March 24, 1934), 353-8. 

“The ‘Teaching Degree,” School and Society, XXXV (January 
30, 1932), 135-40. Discusses the problems involved in offering graduate 
degrees for teachers. 

Hersurn, S. B. “Teacher or Researcher—Which? Another Point of View,” 
Phi Delta Kappan, XVI (October, 1933), 93-4, 100. 

Herssr, R. L. “Bureau of Research in Public School Systems with Ref- 
erence to Cities of 100,000 Population or Less.” Unpublished Doctor’s 
thesis, Harvard University, 1930. 
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“The Functions of Bureaus of Research,” Journal of Educational 
Research, XXIV (December, 1931), 372-6. Based on returns from 
some sixty bureaus in cities of 100,000 population or less. 

Hiatt, L. R. “Provisions Made in Teachers Colleges for Graduate Work,” 
Educational Administration and Supervision, XVI (October, 1930), 
507-18. A catalogue analysis of public teachers’ colleges. 

Hicsrr, E. C. “Protect the Ph.D.” Educational Administration and 
Supervision, XIV (November, 1928), 555-8. Urges the desirability of 
a teaching graduate degree for those who have neither the capacity 
for, nor the interest in, a research degree. 

Hit, C. M. “Progressive Procedures in Graduate Study,” Progressive 
Education, XII (May, 1035), 352-7. Describes the Yale plan of 
graduate study. 

Hunts, E. C. “Foreign Language Requirements for the Doctor of Philoso- 
phy,” Bulletin of the American Association of University Professors, 
XIV (March, 1928), 186-9. States the practices at various institutions. 

Homes, H. N. “The Place of Research in the College,” Science, New 
Series, LXVII (June 1, 1928), 539-43. Lists intellectual immoralities, 
ways to promote research in colleges, and functions of the research 
committee. 

Hosic, J. F. “Appropriate Graduate Work for Elementary School Prin- 
cipals,” Teachers College Record, XXXIII (October, 1931), 45-51. 
“Qualification for Research,” Journal of Educational Method, 
VIII (February, 1929), 247-8. Emphasizes the need for mastery of 
both the subject-matter field and the technique of research involved 

in an investigation. 

“Science or Art?” Educational Method, VIII (May, 1929), 433- 
4. Emphasizes the danger in neglecting the art of teaching in the 
current scientific movement in education. 

Hower, C. E. “The Functions of a Research Department,” American 
School Board Journal, LXXXV (November, 1932), 50, 64. 

Hunt, R. D. “The Doctor of Philosophy Degree,” School and Society, 
XXIII (January ọ, 1926), 31-7. Discusses a number of problems of 
the graduate school, including residence requirements, preliminary 
examination, dissertation, language examinations, and final oral exami- 
nation. 

Hunter, F. M. “What Does the Superintendent Expect of the Research 
Department?” Journal of Educational Research, XVII (May, 1928), 
324-34. Discusses the functions of a research department, including 
appraisal, measurement, investigation, experimentation, and adminis- 
tration of highly specialized fields. 

Hunter, W. L. “The Value of the Master’s Degree,” Industrial Educa- 
tion Magazine, XXXVI (September, 1934), 201-2. 

Jacx, June. “A Rejoinder to ‘Instruction in Our Graduate Schools—A 
Protest,’ ” School and Society, XXXIV (October 17, 1931), 535-6. 
Jounson, E. H. “The Ministers’ Degree,” School and Society, XXXIII 

(March 14, 1931), 371-2. 

Jounson, P. O. “The Benefactions of Philanthropic Foundations and Who 
Receive Them,” School and Society, XXXV (February 20, 1932), 
264-8. 
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Jones, A. J. “The Superintendent of Schools and Research,” Educational 
Administration and Supervision, XV (February, 1929), 81-92. Dis- 
cusses research from the following angles: meaning, method, steps, 
value to superintendent, etc, 

Junes, C. L. “University Training for Government Work in Economics 
and Statistics,” University Training for the National Service, pp. 
273-80. Proceedings of a conference held at thé University of Minne- 
sota, July 14-17, 1931. Minneapolis, Minn.: University of Minnesota, 
1932. 

E A. N. “The Necessity for Research in Education,” American 
School Board Journal, LXXTII (August, 1926), 41-2. 

joop, C. H. “A National Program of Educational Research,” Educational 
Record, XI (July, 1930), 164-72. Describes the possibilities for research 
of the Committee on Educational Problems and Plans of the American 
Council on Education. 

“Educational Research and the American School Program,” Educa- 
tional Record, IV (October, 1923), 165-77. 

—— “How Can American Educational Forces Coöperate More Effec- 
tively?” School and Society, XXXIII (March 14, 1931), 351-6 
Through strenuous effort, differentiation of functions and sequences, 
vision, and intelligent endeavor. 

“The Place of Research in a Program of Curriculum Develop- 

ment,” Journal of Educational Research, XVII (May, 1928), 313-23. 

“The School of Education,” Higher Education in America, Chapter 
VI. Edited by R. A. Kent. Boston: Ginn and Co., 1930. 

— “The Systematic Organization of Courses in Education,” School 
Review, XLV (October, 1933), 575-84. 

Kearney, N. C. “Teacher or Researcher—Which?” Phi Delta Kappan, 
XVI (August, 1933), 67, 72. 

Kerauver, G. N. “A Challenge to Educational Leadership,” Phi Delta 
Kappan, XVII (April, 1935), 177-80. 

Kerry, R. L, and Anperson, R. E., “Great Teachers and Some Methods 
of Producing Them,” Journal of Educational Research, XX (June, 
1929), 22-30, Based on inquiries sent to 349 representative college and 
university executive officers, 

Kemmerer, W, W. “Does Research Bring Disrespect?” School Executives 
Magazine, LI (February, 1932), 258-9. 

Keys, Noel. “Research, Service, and Leadership in Education,” School and 
Society, XXXVIII (October I4, 1933), 490-5. 

Kirk, R. E. “Research for the Undergraduate,” Journal of Chemical Edu- 
cation, IX (February, 1932), 280-5. 

Karson, H. D, “A Further Study of the Turnover of Psychologists,” 
Journal of Applied Psychology, XVII (October, 1933), 542-9. 

Kunxer, B. W. “The Representation of the Colleges in Graduate and 
Professional Schools,” School and Society, XXV (March 12, 1932), 
364-70. 

*Latne, G. J. “The Doctor of Philosophy and College Teaching,” Associa- 
tion of American Colleges Bulletin, XVI (March, 1930), 95-113. Points 
out the rare combination of qualities which make for most successful 
college teaching. 
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Larson, E. L. “Major Problems Relating to the Master’s Degree in Educa- 
tion,” Educational Administration and Supervision, XVIII (December, 
1932), 699-711. 

Loomis, C. P, “The Group Method in Rural Studies,” Sociology and Social 
Research, XX (November-December, 1935), 127-35. Discusses how 
a group of students may study family and village life coéperatively. 

Lourtit, C. M., and Patricx,,J. R. “A Study of Students’ Knowledge in 
the Use of the Library,” Journal of Applied Psychology, XVI (Oc- 
tober, 1932), 475-85. 

Loorsourow, G. C. “Leadership: A Pragmatic Substitute for Authority,” 

h Phi Delta Kappan, XIII (February, 1931), 130-3. 

Lowett, A. L. “The College and the Graduate School,” Educational 
Record, IX (January, 1928), 3-8. Suggests that candidates for the 
Doctor’s degree should not be required to take the Master’s degree, 
since they may be delayed and diverted in attaining their real objective 
by doing so. 

McCarr, W. A., and Rucer, G. L. “The Reliability of a Ph.D. Research 
Dissertation in Educational Psychology,” School and Society, VII 
(April 13, 1918), 441-9. 

McMurry, F. M. “The Biggest Thing in Teaching,” Teachers College 
Record, XXVIII (November, 1926), 215-9. Emphasizes friendship and 
close personal relations with students as the most important thing 

i in teaching. 

Marone, Kemp. “English Linguistics and the Ph.D.,” English Journal, 
College Edition, XVIII (April, 1929), `312-20. Shows that modern 
English literature is studied in Doctors’ dissertations rather than 
linguistics and Old English. 

* MacNerL, J. R. Admission of Students as Candidates for the Master’s 
Degree. Teachers College Contributions to Education, No. 524. New 
York: Bureau of Publications, Teachers College, Columbia University, 
1932. Pp. viii + 91. 

Manwartino, W. H. “Our Contemporary Research ‘Aces,’ ” Science, LXXI 
(April 4, 1930), 361-2. Comments on the contribution of the large 
number of unexploited contemporary medical scientists as compared 
with the spectacular performances of so-called geniuses. 

Martens, E. H. Organization of Research Bureaus. Bureau- of Education 
City School Leaflets, No. 14, 1924. Washington: Bureau of Educa- 
tion. Pp. 16. 

Maruews, C. O. “The Introductory Course in Educational Measure 
ments,” Educational Administration and Supervision, XXI (September. 
I I-47- 

Hee E seta the Master’s Degree be Conferred Honoris Causa?” 
School and Society, XXIII (April 17, 1926), 502-4. _ 

Meram, J. L. “Graduate Courses in Education,” Educational Adminis- 
tration and Supervision, XVII (November, 1931), 574-82. Covers 
state universities. 

Merritt, W. A. “The Ph.D. Fetish,” School and Society, XXX (Decem- 
ber 14, 1929), 813. Suggests that those who criticize the Doctor's 
degree as a qualification for teaching fail to consider that this is 
not the only qualification involved in making an appointment; 
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a majority of desirable positions are filled from the ranks of Ph.D.’s, 
although not all the holders of such a degree receive appointments. 

Murer, J. C. “Educational Administrators Must Be More Highly 
Trained,” Nation’s Schools, XII (November, 1933), 37-40. Suggests 
the type of training which should be given by the school of education 
to principals and superintendents. 

Mutter, L. W. “Professional Talents and Faculty Assignments,” School 
and Society, XXXII (November 29, 1930), 729-31. Urges that work 
be assigned according to ability, whether in administration, research, or 
teaching. 

Monroe, W. S. “Service of Educational Research to School Adminis- 
trators,” American School Board Journal, LXX (April, 1925), 37-9, 
122, 125. 

Moran, George. “Instruction in Our Graduate Schools—A Protest,” 
School and Society, XXXIV (July 11, 1931), 58-9. A criticism of 
graduate work in literature. 

* Mort, P. R. “Organization for Effective Educational Research in Colleges 
and Universities,” Teachers College Record, XXXVI (April, 1935), 
541-58. 

Netson, R. W. “Theaetetus Ph.D.,” Journal of Higher Education, IV 
(May, 1933), 234-40. Criticism of restrictions placed upon doctorate 
candidates in working out their programs and theses. 

Newcoms, T. M., and Watson, Goodwin. “An Experiment in Having 
Graduate Students of Education Score Their Own Test Papers,” 
Educational Administration and Supervision, XVI (January, 1930), 
29-33. Involved 238 students in educational psychology; reports a 
total of fifty-one papers raised in score from one to eight points 
by students. 

Newton, J. H. “A Background for Practical Research,” Phi Delia Kappan, 
VIII (April, 1926), 1-6. 

“What Research Can Do for the Superintendent,” Journal of 
Educational Research, VIII (September, 1923), 106-12. 

Norr, V. H. “Measuring Scientific Thinking,” Teachers College Record, 
XXXV (May, 1934), 685-93. 

“Teaching the Habit of Scientific Thinking,” Teachers College 

Record, XXXV (December, 1933), 202-12. 

“The Habit of Scientific Thinking,” Teachers College Record, XXV 
(October, 1933), 1-9. 

O'Brian, R. E. “The Minister’s Degree,” School and Society, XXXIII 
(January 17, 1931), 87-9. Discusses degrees appropriate for graduate 
work in religion. 

“The Preacher’s Degree,” Religious Education, XXVII (January, 

1932), 66-9. 

Opam, G. A. “Research in State Teachers Colleges and Normal Schools 
of the United States.” Unpublished Doctor’s thesis, New York Uni- 
versity, 1932. 

Ocpen, R. M. “The Relation of Undergraduate to Graduate Study,” 
School and Society, XLIII (February 1, 1936), 137-43. 

* Occ, F. A. Research in the Humanistic and Social Sciences. New York: 
The Century Co., 1928. Pp. 454. 
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Pace, E. A. “Does Research Interfere with Teaching?” The Effective 
College, Chapter XI. Edited by R. L. Kelly. New York: Association 
of American Colleges, 1928. 

“Does Research Interfere with Teaching?” Association of American 
Colleges Bulletin, XIII (April, 1927), 167-74. Concludes that research is 
the first consideration of the professor in the graduate school, and 
that the professor who conducts graduate work, as well as teaches 
undergraduates, has a combination which requires research, However, 
he considers teaching as the primary function of the instructor in the 
separate college; only when teaching skill has been demonstrated and 
there is assurance of continued effectiveness in teaching should research 
be undertaken. 

Parrrey, T. R. “The Master’s Degree in Modern Foreign Languages,” 
Modern Language Journal, XIX (April, 1935), 489-500. 

Pecustet, L. A. “Seminars for Graduate Study,” Journal of Higher Edu- 
cation, VII (January, 1936), 23-8. 

Pecustein, L. A., and Others. “The Cincinnati Seminar Plan,” School and 
Society, XLI (June 1, 1935), 736-9. 

* Pressey, S. L., Pressey, L. C., and Others. Research Adventures in Uni- 
versity Teaching, pp. 140-7. Bloomington, Il.: Public School Publish- 
ing Co., 1927. Studied current opinions relative to the requirements 
for the Doctor’s degree by circularizing 214 recipients of Ph.D. de- 
grees who belonged either to the American Psychological Association 
or the Educational Research Association; majority of time went to 
teaching; many made no use of foreign languages in their work, and 
considered changes desirable in requirements for the Ph.D. degree 
in terms of the foreign-language requirement, adaptation of training 
to future needs, and greater freedom from detailed requirements. 

Ranxiy, P. T. “How an Instructional Research Department Can Assist 
Teachers,” Journal of Educational Research, VIII (October, 1023), 
187-98. 

Reep, Anna Y. The Effective and the Ineffective College Teacher. New 
York: American Book Co., 1935. Pp. xiv +344. Considers graduate 
study and research in relation to effective college teaching. 

Reever, W. G. “Making Educational Research Function,” American 
School Board Journal, LXXXIII (November, 1031), 29-30, 92. 
Ricwarpson, L. B. “Training for College Teaching,” Journal of Higher 

Education, 1 (November, 1930), 425-35- 

* Ropertson, D. A. “The Graduate School—Its Functions and Criteria,” 
Problems of College Education, pp. 164-76. Edited by Earl Hudelson, 
Minneapolis, Minn.: University of Minnesota Press, 1928. 

Roerzincer, F. X. “The Graduate Student Looks at the Professor and 
Graduate Study,” English Journal, College Edition, XXIV (January, 
1935), 55-8. 

Rocers, L. B. “Honorary Degrees for Distinction in the Field of Educa- 
tion,’ School and Society, XXXII (September 6, 1930), 325-6. Based 
on a canvass of the practice in leading graduate institutions. 

Rocers, L. B. “The Place of Schools of Education in the Program of 
Improvement of University Instruction,” School and Society, XXXVI 


(August 27, 1932), 257-64. 
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Ross, C. C. “Relation of Leadership to Research and Service,” Phi Delta 
Kappan, XII (April, 1930), 188. k 

Ruepicer, W. C. “English and the Ph.D.,” School and Society, XXX 
(November 30, 1929), 742-3. Suggests that the science type of research 
is not entirely appropriate for English and that themes for poems, 
dramas, novels, short stories, and essays may serve the same purpose 
in English as the so-called scientific thesis in certain other fields of 
learning, 

Rucc, Harold. “The American Experimental School,” Teachers College 
Record, XXX (February, 1929), 407-24. Emphasizes the need for 
greater scholarship and a more truly experimental attitude of mind. 

Russert, J. E. “Teachers College and Research,” Teachers College Rec- 
ord, XXXIII (November, 1031), 92-5. 

Russert, W. F. Report of the Dean of Teachers College for the Year 
Ending June 30, 1934. Teachers College Bulletin, November, 1934. 
New York: Teachers College, Columbia University. Pp. 140. Describes 
a proposed plan for training advanced graduate students, influenced 
by procedures in use at the University of Paris, involving an Ed.D. 
degree at Columbia University. 

Sanoren, P. V. “The Participation of the Classroom Teacher in Educa- 
tional Research,” Educational Administration and Supervision, XV 
(November, 1929), 593-601. Discusses the place of research in educa- 
tion, values of teacher participation in research, and practicability of, 
and overcoming difficulties of teacher participation in, research. 

*Scates, D. E. “Training for Research in the Light of Field Requirements,” 

Elementary School Journal, XXXIII (October, 1932), 96-105. Points 
out that in addition to research, four types of routine service are 
commonly rendered by research bureaus: reports; testing programs; 
construction of the curriculum, of tests, and of instructional plans; 
and reference work, including bibliographies and summaries. 

Sears, J. B. “The Administration of Public School Research Policies,” 
Journal of Educational Research, XXVI (November, 1932), 186-98. 

“The Function of Research in Curriculum Making,” California 

Quarterly. of Secondary Education, VIII (January, 1933), 147-53- 

Seasnore, C. E. “Printer’s Copy for Master’s Thesis,” Journal of Higher 
Education, III (March, 1932), 146-7. 

Sxav, O. K. “The Ph.D. Fetish,” School and Society, XXXI (March 20, 
1930), 432. Comments on the overemphasis, even in India, of formal 
qualifications in terms of academic degrees, to the neglect of additional 
practical qualifications for teaching. 

Sricuter, C. S. “ ‘Debunking’ the Master’s Degree,” North Central Asso- 
ciation Quarterly, III (March, 1929), 463-7. Lists suggested standards 
for the improvement of work for this degree, including certain re- 
strictions op credit for summer, extension, and correspondence work. 

Situ, D. E. “Necessary Steps in the Solution of Certain Problems in 

_ American Schools of Education,” Teachers College Record, XXXVI 
(April, 1935), 566-92. 

Samira, F. M. “The Ph.D. and the Research Fetish,” School and Society, 
XXX (October 26, 1929), 559-62. Discusses the pettiness of many 
research studies and theses in the field of literature. 
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Saatn, H. L. “The Growing Demand for Research Workers in Bureaus 
of Educational Research,” Eighteenth Annual Conference on Educa- 
tional Measurements, pp. 33-47. Bulletin of the School of Education 
of Indiana University, Vol. VII, No. 5. Bloomington, Ind.: Indiana 
University, 1931. 

Swennen, David. “Aspirational Notions of Leadership,” School and Society, 
XXXI (May 17, 1930), 661-4. A criticism of indiscreet use of the 
term, leadership. 

*Spanr, W. E., and Swenson, R. J. Methods and Status of Scientific 
Research, Chapters XV-XVIII. New York: Harper and Bros., 1930. 

Srenquist, J. L. “Better Schools Through Research,” Baltimore Bulletin 
of Education, 1X (October, 1930), 25-7. Discusses the need for in- 
formation concerning what to teach and how to teach it. 

“The Work of the Bureau of Measurement, Statistics, and Research 
(Baltimore),” Baltimore Bulletin of Education, XII (November, 
1933), 4-9. 

Sropparp, G. D. “The Lag Between Practice and Research,” Phi Delta 
Kappan, XVII (April, 1935), 180-2. 

*Srratton, Dorothy C. Problems of Students in a Graduate School of 
Education. Teachers College Contributions to Education, No. 550. 
New York: Teachers College, Columbia University, 1933. Pp. x + 168. 

Srraver, G. D. “The Scientific Approach to the Problems. of Educational 
Administration,” School and Society, XXIV (December 4, 1926), 
685-95. 

SrricKLanp, V. L. “Causes of Waste in Education: Reply to B. R. Buck- 
ingham,” School and Society, XXV (January 1, 1927), 15-6. Discusses 
reasons for failure of learning to function in after-school life. 

Symonps, P. M. “A Course in the Technique of Educational Research,” 
Teachers College Record, XXIX (October, 1927), 24-30. 

‘Tuetsen, W. W. “The Function and Value of a Public-School Research 
Bureau,” American School Board Journal, LXXVIII (April, 1929), 
47-8, 142. Discusses the task, types of service, division of functions 
between bureaus, and improvement of research bureaus. 

Tuom, W. T., and Fern, R. M. “The Achievement of Geology through 
Coöperative Research,” Science, LXXII (August 1, 1930), 117-8. 
TuornptkE, E. L. “The Value of Research in Education,” Teachers College 

Record, XXXIII (November, 1931), 96-9. 

Toors, H. A. “The Selection of Graduate Assistants,” Personnel Journal, 
VI (April, 1928), 457-62. Deals with the principles which should 
obtain in evaluating evidences of occupational fitness. 

Toors, L. C. “Research for Teachers in Training,” Teachers College 
Record, XXXIV (December, 1932), 228-34. A plan for coérdinating 
codperative and individual research, applied in a research program for 
a class of prospective teachers in nursery schools. 

Traxier, A. E. “The Educational Status of Phi Delta Kappans,” Phi 
Delta Kappan, XIV (August, 1931), 50. 

Townsenp, Marion E. “The Organization and Function of Public-School 
Departments of Research in Cities in the U. S. of 100,000 and over.” 
Unpublished Master’s thesis, Teachers College, Columbia University, 
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Trow, W. C. “Dangers of the Doctorate,” Educational Review, LXIX 
(January, 1925), 35-8. Deplores the narrowing influence of graduate 
specialization and the dissertation. 

* Torts, J. H. “The Graduate School,” Higher Education in America, 
Chapter XI, Edited by R. A. Kent. Boston: Ginn and Co., 1930. 
Tyrer, R. W. “Ability to Use Scientific Method,” Educational Research 
Bulletin, XI (January 6, 1932), 1-9. Describes a series of tests to 
determine how well students at Ohio State University have learned 

to apply scientific method. 

Upnam, A. H. “Educating College Teachers,” Association of American 
Colleges Bulletin, XVI (March, 1930), 75-83- Describes a number of 
plans for the improvement of college teaching; comments on the 
relation between good teaching and research. 

Ursuaxt, C. C. “Responsibilities of Research Workers in Teachers Col- 
leges, School and Society, XLIII (February 15, 1936), 223-26. Prac- 
tical suggestions for stimulating research on the part of faculty 
members. 

VOELKER, P. E., and Others. “A Program of Demonstration and Research,” 
Educational Record, XVI (April, 1935), 207-16. Presents a plan for 
demonstration and experimentation in education throughout the State 
of Michigan. 

Watxer, Helen M. “College Instruction in Statistics, 1933-1934,” Teachers 
College Record, XXXVII (March, 1936), 527-39. 

Watxer, Helen M. “Problems in the Training of Research Workers,” 
Journal of Educational Research, XXVI (February, 1933), 442-8. 

Watson, Goodwin. “The Demand for Psychological Counselors in the 
Public Schools,” School and Society, XXXIII (June 27, 1931), 871-2. 

Weiwemann, C. C. “Research Bureaus Should Be Centered in School 
Organization,” Nation’s Schools, XII (September, 1933), 27-30. 

, and Rucs, C. E. “Professionalized Research in Education,” Phi 
Delta Kappan, XVI (October, 1933), 77-8. 

Werp, L. D. “The College Teacher and Research,” Science, LII (July 16, 
1920), 45-8. Emphasizes the value of research to the teacher, even 
in the smaller institutions. 

Werzez, W. A., and Asupaucn, E. J. “The Work of the National Com- 
mittee on Research in Secondary Education,” School and Society, 
XLI (April 27, 1935), 580-2. 

Wuirrtr, G. M. “The Improvement of Educational Research,’ School 
and Society, XXVI (August 27, 1927), 249-59. 

Warre, N. I. “Teaching Versus Research,” School and Society, XXV 
(January 23, 1932), 109-13. 

Wuer, R. L. “The Educational Mill,” Progressive Education, VII (May, 
1930), 167-9. A discussion of the need for better teaching as well as 
for fundamental research, 

Warnms, S. R. “Strategical Periods for the Encouragement of Research 
Students,” Science, New Series, LXVIII (September 14, 1928), 244-5. 
Within or immediately after college days. 

Wrruers, J. W. “Professional Education of Directors of Research in 
City School Systems,” Journal of Educational Research, XVII (May, 
1928), 335-44. Lists type of work done by research bureaus, training 
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of directors of research, and principles in the organization of research 

bureaus. 

“The Scientific Method in the Study of Education,” Journal of 

Educational Research, XXI (March, 1930), 212-20. Discusses in 

detail the possibilities for research in public-school systems, bureaus 

of research, and the qualifications of the director of research. 

“The Scope and Function of a School of Education,” Journal of 
Educational Sociology, V (May, 1932), 538-46. 

Woon, W. E. A Study of Performance on the Graduate Level Based 
upon Subject Marks and Intelligence Test Scores: Ages 20 to 60. 
Danville, Ind.: Published by the author, 1931. Pp. 56. 

Woopy, Clifford. “Specifics Versus Generalities for Field Workers,” Journal 
of Educational Research, XXIX (January, 1936), 372-3- Urges that 
definite problems, with suggested techniques of investigation, be 
brought to the attention of field workers. 

Woopy, Clifford. “The Values of Educational Research to the Classroom 
Teacher,” Journal of Educational Research, XV1 (October, 1927), 
172-8. 

*Wricut, Edith A. Organization and Functions of Research Bureaus in 
City School Systems. Office of Education Leaflet, No. 2, 1931. Wash- 
ington: Office of Education, 1931. Pp. 14. 

Wvxorr, G. S. “On the Revision of Ph.D. Requirements in English,” 
English Journal, College Edition, XVII (March, 1928), 213-20. Urges 
that both teaching and research degrees be offered in English, and 
suggests the requirements for the teaching PhD. degree in English: 
(1) three years of satisfactory graduate study, (2) no thesis, (3) in- 
tensive survey courses of the whole field of English and American 
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research in English and the attitude the college teacher of English 
should assume toward doing research. 

Zook, G. F. “The Present Position of Graduate Studies in the United 
States,” School and Society, XLIII (January 11, 1936), 41-9. 

“The Adventures of Alice through the Looking Glass: The Graduate 
Formula,” School and Society, XXXIX (January 20, 1934), 84-6. 
‘A caustic criticism of non-essential routine and formalism. 
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CHAPTER XVI 


SUMMARY 


To synthesize the conclusions reached in preceding chapters, 
a brief final statement seems in order; indeed some workers 
may prefer to read this chapter first as an overview of or in- 
troduction to the entire volume. 

The nature of scientific thinking. In his search for truth, 
man came to utilize the methods of science rather late in the 
history of civilization. Earlier approaches in interpreting 
natural phenomena were through supernatural beliefs, the 
voice of authority, syllogistic reasoning, and empirical observa- 
tions. Scientific method involves both inductive and deductive 
modes of thinking, with emphasis on facts, analysis, hypotheses, 
generalization, freedom from bias, and the use of measurement. 
These characteristics of science have been stated in other 
words as involving mathematical precision and accuracy, ob- 
jectivity, verifiability, impartiality, and expertness.’ The steps 
in scientific procedure are essentially the same as those em- 
ployed in good thinking or problem solving, namely, selection 
and definition of the problem, a survey of pertinent related 
information, formulation and development of a hypothesis, col- 
lection of data, analysis and summarization of data, and for- 
mulation of conclusions and generalizations. However, it is 
granted that the human mind does not always work in such an 
orderly sequence, but may weave back and forth “shuttle-like.” 
In a flash of insight, the genius may jump intervening steps to 
a tentative solution of his problem and later work more slowly 
through the detailed proof of his conclusions. The chapters of 

18. C. Parker, General Methods of Teaching in Elementary Schools, pp. 107-11. 


Boston: Ginn and Co., 1922. 
E. L. Thorndike, Principles of Teaching, pp. 265-8. New York: A. G. Seiler, 1906. 
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this book are arranged approximately in terms of the sequence 
of steps followed in scientific procedure. 

Selection and definition of the problem. A major difficulty 
of many educational workers lies in recognizing the problems 
which are plainly discernible to the investigator trained to see 
such issues. In Chapter II a detailed analysis was made of 
ways in which this “problem blindness” may be corrected, at 
least in part. Actual selection of the problem involves consider- 
ation of such factors as novelty, the investigator’s interest, 
practical value, the worker’s qualifications, availability of data, 
cost, and time required. After the problem (usually first ex- 
pressed in general terms or even representing an entire field 
of investigation) has been selected, it must be more definitely 
formulated and stated, as well as defined and delimited. 

Survey of related information. The canvass of information 
and literature related to the problem chosen probably has been 
progressing simultaneously with the actual selection of the 
problem for investigation. As a matter of fact, this survey of 
related information may accomplish any one or all of five 
purposes; it may (1) show whether the evidence already avail- 
able solves the problem adequately without further investiga- 
tion; (2) provide ideas, theories, explanations, or hypotheses 
valuable in formulating the problem; (3) suggest methods of 
research appropriate in attacking the problem; (4) locate com- 
parative data for use in the interpretation of results and in 
the formulation of conclusions; and (5) contribute to the 
scholarship of the investigator. The major types of educational 
literature, in terms of the kinds of thinking or research pro- 
cedures inyolved, from which such assistance may be secured, 
are: (x) statistical or experimental studies with emphasis on 
objective methcds and mathematical statement of results; (2) 
theoretical, speculative, or philosophical discussions with em- 
phasis on the methods of logic and rationalization; and (3) 
historical investigations, based in the majority of cases on 
documentary evidence. However, library materials and guides, 
as well as the printed sources, are not classified in such terms. 
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_ Therefore, a more practical working classification, as described 
at length in Chapter III, includes: (1) periodical literature, in 
journals; (2) printed material in bound form, such as books, 
monographs, year-books, and bulletins; (3) graduate theses, 
usually unpublished, except for a limited number of Doctors’ 
dissertations; and (4) certain miscellaneous sources of infor- 
mation. 

Use of hypotheses. Once a problem has presented itself or 
has been sought out by the investigator, one of the distinguish- 
ing characteristics of scientific method is the tendency to 
formulate a shrewd guess explaining the phenomenon under 
observation. That is, the human mind in many cases does not 
proceed systematically through a given sequence of steps or 
wait for the complete enumeration of all the facts before 
formulating a hypothesis explaining tentatively the situation 
under investigation. Of course, if the first hypothesis proves 
fallacious when checked against the observed facts, another 
one may be formulated and tested until the correct explanation 
is found. Such a preliminary or tentative concept gives direc- 
tion to the investigation until proved or disproved, or until 
displaced by another hypothesis. The formulation and testing 
of hypotheses are treated in an introductory way in Chapter I, 
and more fully in Chapter IV. 

Coliection of data. Obviously, a given problem may be 
solved, or a hypothesis tested, only by the collection of per- 
tinent data. The organization of Chapters V-X is in terms of 
methods of collecting data or ways of attacking educational 
problems. 

Historical research examines the evidence of the past, which 
may be used as an aid in analyzing and interpreting the pres- 
ent, or even the future. Its major processes are: (1) collection 
of data, with due consideration of documents and relics (or 
remains), of primary and secondary sources, of bibliographical 
procedure, and of an adequate system of organizing data; (2) 
criticism of data collected, including the processes of external 
(or lower) criticism and internal (or higher) criticism; and 
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(3) presentation of the facts in readable form, involving prob- 
lems of organization, composition, exposition, and interpreta- 
tion. 

Normative-survey research involves a canvass of present 
practice or conditions, or the establishment of norms or cen- 
tral tendencies, utilizing extensively such data-collecting in- 
struments as questionnaires, tests, check-lists, rating scales, 
and score cards, as well as interviews, activity analysis, text- 
book analysis, etc. These instruments are not of themselves 
major approaches to the solution of educational problems, but 
serve as devices for the collection of data. 

As has been suggested in the preceding paragraph, data- 
collecting instruments, such as are discussed in Chapters VII 
and VIII, are used most extensively in the normative-survey 
type of research, although standard tests are useful in experi- 
mental research for the equation of groups and for the meas- 
urement of gains. Data collected by such instruments also are 
valuable in making case studies and certain other types of 
research involving causal relationships. 

The six major types of normative-survey research employ 
(1) survey-testing, (2) questionnaire, (3) documentary- 
frequency, (4) interview, (5) observational, and (6) survey- 
appraisal procedures, including school surveys. 

Experimental research, under controlled conditions, with 
manipulation of only a single variable, tries out in classroom 
or laboratory different educational plans or procedures. Most 
classroom experimertation employs group methods (one-group, 
equivalent- or parallel-group, and rotation-group methods). 
Major considerations in experimentation are definition and re- 
striction of the experimental factor, specification as to the 
nature and scope of the pupil achievement to be measured, 
control of non-experimental factors, selection or construction 
of adequate measuring instruments, extension of the experi- 
ment over a sufficient period of time, and the reliability of the 
obtained difference between gains of the experimental and con- 
trol groups or individuals involved. In the laboratory type of 
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experimentation, conditions may be more carefully controlled, 
suitable apparatus and equipment provided, subjects studied 
individually, and a more minute analysis of such problems as 
learning worked out. 

Certain other types of investigation, sometimes termed 
causal research, are especially adapted to studying causal rela- 
tionships; they observe an effect and seek to determine its 
cause or pertinent relationships through a study of likenesses 
and differences (the causal-comparative method), case-study 
and genetic methods, diagnostic and prognostic procedures and 
instruments, or correlation techniques. 

Analysis and interpretation of data, and generalization. 
It is in the classification, analysis, summarization, and interpre- 
tation of data that statistical method proves especially valu- 
able, including the measures of central tendency, variability, 
correlation, etc. Statistical procedures are also used extensively 
as aids in all of the previously mentioned types of research 
(experimental, survey, and causal), with the exception of his- 
torical investigations. Statistical method is considered a way 
of manipulating data rather than a technique for securing new 
information, although occasionally an original contribution to 
the field by way of a new formula, statistical device, slide rule, 
or calculating machine is forthcoming. Of course, statistical 
calculations do not take the place of, but are supplementary 
to, good thinking and sound logic. 

In this volume, educational philosophy is considered, not 
an independent method of collecting data, but a valuable aid 
in setting up the problem, formulating and testing hypotheses, 
interpreting data, generalizing, and in developing standards 
for evaluation. 

Preparation of the report, and standards for evaluation of 
research. It is recognized that the work of writing up the report 
and the development of standards for critical evaluation of 
research are not deferred in the actual process of investigation 
until the study is completed. In fact, the writing, at least in 
the form of a first draft of notes, may have been progressing 


774 EDUCATIONAL RESEARCH 


advantageously in connection with each step of the research 
process. Likewise, it is hoped that critical standards were being 
developed at the same time. The student of educational prob- 
lems should adopt, adapt, or develop sound criteria for evalu- 
ation, which ultimately become a part of his own professional 
and scientific equipment: (1) in choosing from the large 
volume of available current literature, (2) for guidance in the 
preparation of his own reports, (3) for analysis of practices 
and innovations in education, and (4) in general for the direc- 
tion of his own educational thinking. 

Training and supervision of research workers. The point 
of view maintained in this volume is that educational workers 
(teachers, administrators, and supervisors) should be so trained 
as to contribute to the solution of problems in their respective 
areas of responsibility: (1) all as consumers of research, (2) 
many as participants in the problem-solving approach because 
of the resulting training value and the increased understanding 
of the educative process, and (3) a more limited number as 
real contributors to the existing store of educational knowl- 
edge. This statement is made with full recognition of the fact 
that an ideal balance between investigational procedure, or 
problem solving, and other elements of the training program 
for field workers probably has not yet been discovered and cer- 
tainly has not been effected in practice. It is also conceded 
that, for many who might otherwise attack their problems 
more scientifically, there are obstacles in the form of heavy 
teaching or administrative loads, as well as a lack of facilities 
for research. However, the real scientist is possessed by a con- 
suming intellectual curiosity which sometimes drives (or leads) 
him forward to the neglect of his own physical comforts and 
even of life itself. Without recommending that one make any 
such extreme of self-sacrifice, it may well be emphasized that 
in the properly qualified educational worker this dual responsi- 
bility of teaching, or administration, or supervision on the one 
hand, and of problem solving or search for truth on the other, 
involves no essential incompatibility. The open-mindedness 
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and the demand for evidence which such a person brings to 
classroom, laboratory, or conference are indeed stimulating in- 
fluences and an indication that gradually there is being estab- 
lished at least a firm foundation for a science or practice of 
education among the million teachers of this country. 


APPENDIX I 
NEEDED RESEARCH IN EDUCATION + 


A. HIGHER EDUCATION, TEACHER TRAINING, AND 
ADULT EDUCATION 


Before enumerating detailed lists of problems which require 
study at the level of higher education, comment may be made 
. concerning the marked increase in attention given educational 
problems in higher institutions. Current interest in problems 
of curriculum reorganization, methods of teaching, and ad- 
ministrative procedure at the college and university level is to 
a considerable extent the result of a rapid increase in student 
enrolment since the late war. This influx of a less select student 
body, with a wider range of individual differences, needs, in- 
terests, and capacities, and the resulting expansion of a limited 
curriculum and modification of inflexible methods of teaching 
have introduced perplexing problems. 

A large amount of educational literature concerned with such 
problems has been accumulating. The United States Office of 
Education at one time considered publication of a bibliography 
of some 13,000 titles in the field of higher education. National 
educational and learned associations and foundations have 
been active in the promotion of investigations at the college 
level. In 1935, 92 books, monographs, and bulletins appeared 
in this field, exclusive of publications in adult education. 
Numerous surveys of higher institutions have been conducted. 
To name only a few major studies, there are the Minnesota 
experiments with class size and science teaching, the Land 
Grant College Survey, the Modern Foreign Language Study, 
the Commonwealth Teacher-Training Study, and the National 


1See Chapter II for an introductory statement concerning needed investigations. 
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Survey of Teacher Education. National foundations and socie- 
ties, denominational groups, regional organizations, state gov- 
ernments, teachers associations in subject-matter fields, and 
individual colleges and universities have promoted intensive 
surveys of the higher institutions involved. The educational 
periodicals dealing with college and university problems now 
include Journal of Higher Education, Junior College Journal, 
School and Society, Educational Administration and Supervi- 
sion, Liberal Arts College Bulletin, and Bulletin of the Associa- 
tion of American Colleges. 

The foregoing facts imply that a wealth of material is avail- 
able for use in university courses dealing with problems of 
higher education. During the summer of 1931 twenty-four uni- 
versities offered courses in this field for administrators, teach- 
ers, and graduate students. There is need for study and 
understanding of problems in higher education on the part of 
workers at both college and high-school levels. Secondary- 
school principals and teachers who know the entire educational 
system are able to provide more adequate opportunity and 
guidance for high-school pupils who expect to enter college. 
Such courses are of especial interest to candidates for the 
Doctor’s degree in view of the fact that 75 per cent of this 
group after graduation go into college or university. work. 
Many of these students work out theses concerned with college 
problems. From 1918-1929, 244 Doctors’ theses dealing with 
problems of higher education were completed. In 1930-1931 
181 Doctors’ theses in higher education were under way. 

Does this rather formidable statement of investigational 
activity at the college level mean that most of the problems 
in this field have been solved? Not at all—especially is evi- 
dence lacking concerning the merits of certain teaching and 
curriculum innovations at both graduate and undergraduate 
levels. At the graduate level, in objective terms, what are the 
merits of the Iowa plan of individual or independent study, of 
the Yale seminar organization, or of the relatively new degree 
programs in education as described in the 1931 Yearbook of 
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the National Society of College Teachers of Education? At the 
undergraduate level, on the basis of experimental evaluation, 
what is the evidence concerning Rollins’ workshop or labora- 
tory plan, Cincinnati and Antioch’s codperative or part-time 
program, Harvard and Princeton’s general reading and tutorial 
plans, Stanford’s independent study scheme, Swarthmore’s 
honors courses, Wisconsin’s Experimental College, and Chi- 
cago’s divisional reorganization ? 

Granted that administrative difficulties may seem almost in- 
surmountable and variable factors apparently uncontrollable, 
the challenge to evaluate these plans on an experimental or 
factual basis remains for the student of objective measurement. 
Minnesota’s work with class size and science teaching is a good 
beginning in this experimental program. It must be kept in 
mind that scientific evaluation does not consist of opinions 
collected from parents, students, or teachers with respect to 
the merits of a given plan; neither is it journalistic description 
of the methods in use; it is not even analysis and criticism 
based on observation and painstaking rationalization. Just 
what scientific evaluation does include is a long story that has 
been reasonably well told in current volumes which deal with 
techniques of measurement, experimentation, statistics, and 
educational research, and in other sections of this book.? 

A. S. Barr, “Research and the Professional Education of 


Teachers,” Journal of Educational Research, XXVII 
(January, 1934), 364-366. 


1. How many new teachers per year, subject, and grade can the 
profession absorb? 

2. What is the present status of the unemployment of teachers? 

3. How many of the present unemployed teachers are bona fide 
teachers legally certified to teach and previously employed in the 
profession? 

4. How many of the present unemployed teachers are physically, 
socially, and emotionally qualified to teach? 

5. What are the physical, intellectual, social, and emotional quali- 
ties prerequisite to teaching success? 

2The preceding paragraphs on higher education are adapted from Carter V. Good, 


“Research in Higher Education,” Journal of Educational Research, XXIV (October, 
1931), 231-232. 
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6. What subject combinations are teachers most frequently called 
upon to teach? 

7. How much training should a teacher have? 

8. Recognizing that teachers are expected to exercise a certain 
amount of social and civic leadership, how well qualified is the average 
teacher, employed and unemployed, for this responsibility? 

9. What are the objectives of teacher-training, that is, what are 
the characteristics (knowledges, skills, habits, attitudes, and interests) 
of the ideal teachers? 

1o. How can these qualities be most efficiently inculcated in the 
teaching profession? 

=z. What kinds of curricula produce the most efficient teachers at 
the different age, grade, and subject levels? 

12. How do teachers trained in different sorts of teacher-training 
institutions succeed as teachers? What are their characteristic weak- 
nesses? Their positional history? 

13. Assuming that the institutional training of teachers will need 
to be continued in service, how can the in-service training of teachers 
be made more effective? How may the specific training needs of in- 
service teachers be most effectively determined? How may the instruc- 
tion of teachers in service be made more effective? 

.14. How can teaching ability be more effectively measured? 


H. A. Brown, “Some Unsolved Problems of Teacher Prep- 
aration,” Educational Administration and Supervision, 
XVII (November, 1931), 590-602. 


Suggests that a genuinely scientific approach to the problems of 
teacher preparation needs to be made: 


1. A national survey of the whole field of teacher training to define 
clearly the functions of teacher-training institutions 
(a) What place have county normal schools and high-school 
teacher-training classes in a national plan of teacher training? 
(b) What is the total task of a four-year degree-granting state 
teachers college? 
(c) By what means shall selective admission be accomplished? 
(d) How shall the functions of guidance, counseling, and place- 
ment be organized and conducted? 
(e) Is the teachers college undertaking work that does not come 
within its scope? 
(f) Are teachers college curricula adequate for the large task 
which confronts these institutions? 
(g) To what extent is the present organization of. the teachers 
college adapted to the functions of that institution? 
2. The question of supply and demand of persons legally qualified 
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to teach, indicating ways in which it may be regulated, to stress the 
advancement of requirements for graduation and certification, and to 
emphasize the importance of selective admission to teacher-training 
institutions 

3. Facilities necessary for the best arrangement of practice teach 
ing, for its effective conduct under any given condition. 

(a) To what extent do students profit from instruction in 
methods and technique previous to entering on student 
teaching? 

(b) At what time during training should student teaching be 
done? 

(c) What changes actually take place in a student during the 
experience of student teaching? 

4. Standards, for teachers college, of a more qualitative nature 
which will promote the effective preparation of teachers 

5. Analysis in the field of curriculum-making in public education 
would throw some light upon the question of how teacher training 
might be reconstructed and fundamentally improved 


B. R. Buckingham, “Research in Teacher Supply and 
Demand,” Educational Administration and Supervision, 
XV (April, 1929), 259-268. 


1. A better program of educational guidance for those entering 
teacher-training work 

2. More effective distribution cf supply in relation to demand for 
teachers 

3. Analysis of the processes of teacher training into their essential 
elements 


4. Development of an effective instrumentality for placing and fol- 
lowing up graduates 


W. W. Carpenter, “Problems in Junior College Education,” 
Junior College Journal, V (October, 1934), 13-15. 


Fifty problems in junior college education relating to: 
Articulation with senior college 
Finance 

Organization 

Records and reports 

Location 

Teaching load 

Student failures 

Individual differences 
Recruiting students 

Class size 
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tz, Curriculum 

12, Extracurriculum 

13. Graduation requirements 
14. Guidance 

15. Supervision and improvement of instruction 
16. Extension 

17. Teaching methods 

18. Junior-college functions 
19. Post-graduate students 
20. Tests and measurements 
21. Publicity 

22. Library service 


Jessie A. Charters, “Problems in Adult Education,” Edu- 
cational Research Bulletin, VIII (May <5, 1929), 207- 
211. 


1. Can adults learn? 

2. Will adults learn? 

3. What shall be taught? 

4. How shall adults be taught? 
5. Who shall teach adults? 

W. W. Charters, “Next Steps: A Program of Research in 
Adult Education,” Journal of Adult Education, IL (Oc- 
tober, 1930), 370-375. 

1. Studies of vocabulary 

2. Selection and training of teachers 

3. Tastes and interests of adults 

4. Administrative problems in adult education 


W. W. Charters and Douglas Waples, The Commonwealth 


Teacher-Training Study, pp. 207-2109. Chicago: Univer- 
sity of Chicago Press, 1929. 


Suggestions for research growing out of the Commonwealth Study 
(all specific references in the following list are to the Study): 

1. Objectives of teaching training 

2. How adequately are the objectives covered by the traits and 
activities contained in the report? 

3. To determine to what extent teachers need to be trained in the 
activities as stated on the master list 

4. To check by experimental procedures the validity of the ratings 
obtained from the various professional groups 

5. To define determinants of curricular value other than the four 
rriteria utilized in the present study 
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6. To interpret relationships between the ratings of different pro- 
fessional groups as shown in the Summary Tables 

7. To analyze with reference to the activity list the materials 
found in existing training courses 

8, The most useful means of incorporating traits with other 
curriculum materials 

o. Refine the techniques of collecting trait actions, translating, 
telescoping, and evaluating the traits as described in Chapter II 

10. To determine the traits of most importance to public school 
teachers of all types, those common to all types and those peculiar 
to certain types 

1r. Evaluation of trait actions with reference to the four criteria 
of the study and others 

12, To check the completeness of the data used in constructing 
the master list of activities È 

13. To check the depth of analysis to which certain divisions, 
subdivisions, and sections should be carried for teachers of different 
types 

14. To check the adequacy of the classification 

15. To check the selection of group teachers 

16, To define the objectives of a given training institution 

17. To investigate the assumption that the activities which teachers 
should perform are more readily identified when the activities that 
teachers do perform are known 

18. To determine the relationship between the amount of experi- 
ence in teaching and nature of the teaching position held as determining 
the difficulty of the type activities for the typical teacher 

19. To improve the techniques of Chapter IV, Section 3 and 
Chapter V, Section 3, whereby the fundamental principles, laws, or 
generalizations of a course may be helpfully related to the teaching 
activities governed by such principles 

20, To determine the relative values of typical elements in the 
training curriculum as contributing to the efficiency of teachers in 
service 

21. To determine the nature and sequence of training courses most 
useful to prospective teachers of a given type 


W. S. Deffenbaugh and W. H. Zeigel, Selection and Ap- 
pointment of Teachers, p. 115. Office of Education Bulle- 
tin No. 17, 1932. National Survey of Secondary Education 
Monograph No. 12. Washington: Office of Education, 
1933. 
1. What constitutes teaching success? 
2, How may future success of inexperienced teachers be pre- 
dicted? 
3. How may the success of teachers in service be measured? 
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E. S. Evenden, Summary and Interpretation, pp. 247-253. 
National Survey of the Education of Teachers, Vol, VI. 
Office of Education Bulletin No. 10, 1933. Washington: 
Office of Education, 1935. 


1. To define good teaching in various types of positions, and to 
develop more accurate means of measuring it 
2. To make educational leaders aware of the State’s responsibility 
for developing a long-term plan for the education of its teachers and 
for promoting systematic publicity campaigns to acquaint patrons and 
teachers with its provisions 
3. To obtain greater uniformity in permanent records about the 
education and the employment of teachers 
4. To remove at once the professionally demoralizing lack of ad- 
justment in many States between supply and demand among teachers 
5. To persuade each state to establish dates by which all teachers 
in service whose preparation is below the accepted standard for the 
State would be expected to meet the current standards 
6. To regulate the supply of prospective teachers by raising 
standards through selective admission of students to curricula for 
teachers 
7. To provide large numbers of teachers better and more 
specifically prepared for rural schools 
8. To provide for more and better-prepared teachers for the Negro 
schools in those States in which separate schools are maintained for 
Negroes ` 
9. To develop greater interest in the distinctly professional ele- 
ments in the education of teachers 
to, To make sure that all prospective teachers possess at least the 
“safety minimum” amount of teaching skill before being certified 
Ir. To adjust curricula for teachers to the work of the junior col- 
leges especially in those States in which the minimum standard of 
ean for elementary teachers does not exceed the junior college 
leve 
12, To promote the realization that the schools will be required to 
take a heavier responsibility for the enjoyable and constructive use 
of increased amounts of leisure 
13. To persuade states which have not already done so to adopt 
systems of restricted certification which will indicate very specifically 
the kinds of positions for which the teacher is prepared and which 
will restrict the teacher’s employment to those positions 
14. To develop and maintain in each state lists of institutions 
“approved” for the preparation of teachers for each type of position 
for which the state issues certificates 
15. To adopt standards and regulations which will make sure that 
the courses in all curricula for teachers are taught by staff members 
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who are adequately prepared—scholastically and professionally—and 
who are genuinely interested in education and in the education of 
teachers 

16. To raise standards for the preparation of all teachers and to 
control the supply of new teachers by obtaining from all institutions 
on the approved list—public and private—codperation in developing 
the program and in carrying out its provisions 

17, To improve the community status—prestige—of the teacher 

18, To develop upon a nation-wide basis such an understanding of 
the significance of education that ways and means will be found and 
approved for restoring recent educational losses and for adding new 
educational services 

19. To procure in a majority of the States the legislation or the 
constitutional amendments essential to the reorganization of the state 
departments of education 

20. Each state must maintain a continuing survey of its teaching 
personnel in order to meet its teacher-training obligations wisely. In 
addition to these surveys some agency, national in its scope, such 
as the Office of Education or the National Education Association, 
should conduct periodic studies of phases of teacher-education so that 
the more important aspects of this large problem may be presented 
on a nation-wide basis at least as often as every five years. [At this 
writing, 1936, five years have elapsed since parts of the Survey were 
taade. ] 


E. M. Freeman, “Suggested Program of Investigation Hav- 
ing in Mind the Improvement of Instruction in Science 
at the University of Minnesota,” Problems in College 
Education, Chapter XXXIV. Edited by Earl Hudelson. 
Minneapolis, Minn.: University of Minnesota Press, 1928. 


Suggests investigation of the effect of: 

Amount of laboratory work on achievement 
Amount of recitation work on achievement 
Amount of lecture work on achievement 

Amount of demonstration work on achievement 
Supervised study on achievement 

Size of class on achievement 

Mentality on achievement. 

Reading ability on achievement 

Previous scholarship on achievement 

Previous training on achievement 

Sex on achievement 

12. Instructor on achievement—(a) experience, (b) training 
13. Total time spent in class meetings on achievement 
14. Required readings on achievement (amount) 


Hon : 
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15. Number of tests on achievement 

16, Types of secondary-school work on achievement, (e.g., small 
high school, large high school, city or country, etc.) 

17. Type of lecture on achievement—(a) forcefulness, (b) dramatic 
appeal, or (c) other qualitative effects 

18. Tempo of lecture on results; concreteness and use of illustra- 
tion 

19. Discussion on results 

20, Projects on results 

21. Time of tests on retention, (e.g., “terminal” quizzes) 

22, Review on retention 

23. Note-taking on learning or retention 

24. Amount of reading on learning (little and much) 

25. Homogeneity of group 

26, After-correction of tests on further tests 

27. Length of class or lecture period 

28. Definiteness of aim on results (specific) 

29, Distractions on individuals and class (sensitivity) 

30. Maturity on results 

3. Library work on results; types of library work 

32. Group assignment 

33. Individual instruction 

34- Field trips on instruction 


W. S. Gray and Others, Current Educational Readjustments 
in Higher Education, pp. 149-150. Yearbook Number 
XVII of the National Society of College Teachers of 
Education, Chicago: University of Chicago Press, 1929. 


Unfortunately many important problems in higher education have 
not been subjected to critical analysis or experimentation, such as the 
fundamental differences in the learning activities of students at various 
levels of advancement or the length of time required for curricula 
of various types. With respect to the latter problem, the practice 
prevails of making all curricula conform to artificial standards, such 
as the four-year requirement for the baccalaureate degree, Although 
much has been done to determine student characteristics and the value 
of ability grouping, few studies have been made to determine methods 
by which students may be relieved from additional requirements as 
soon as essential qualitative standards have been attained, Further- 
more, very little has been done to define such standards in either 
academic or professional institutions, 

The curriculum studies which have been made thus far are largely 
summaries of current practice. There is urgent need of critical studies 
which seck to determine basic principles of curriculum construction 
and valid materials of instruction designed to serve specific cultural 
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or vocational needs, Although several studies have been made of the 
merits of different teaching procedures, only an entering wedge has 
been made into this important field. With few expections, experimental 
studies have not been made of the results of the newer methods or 
techniques of teaching, such as the tutorial plan, and reading for 
honors. Too frequently, old methods are discarded and new ones 
adopted without critical studies of the weaknesses of the old or ex- 
periments to determine the merits of the new. The same criticism 
applies to many of the administrative changes which have been made 
during recent years, For example, personnel work has been hailed 
with great enthusiasm by many colleges. Numerous reports have been 
made of personnel activities. Very few studies have been made, how- 
ever, to determine objectively the advantages and limitations which 
attach to them. The examples which have been given emphasize the 
fact that there is urgent need of the wider application of scientific 
methods in the critical study of practically every problem which 
higher institutions face to-day. 


M. E. Haggerty, “Experimenting with the College ‘Teacher’s 
Problem,” Association of American Colleges Bulletin, XV 
(March, 1929), 99-110. 


Believes that ten years of experimental attack upon certain prob- 
lems, as a basis for courses in the teaching of college subjects, are 
needed, especially on: 

1. The problem of college teaching 

2. Techniques of making curricula 

3. Methods of student marking, examinations, and evaluation 


V. A. C. Henmon, “The Function, Value, and Future of 
Educational Research in Colleges and Universities,” Jour- 
nal of Educational Research, XXVII (March, 1934), 
493-502, 


Contributions experimentally tested, using techniques and measuring 
devices that educational science has slowly been developing, are needed 
in the following fields: 

1, What aims or objectives can be realized? 

2. What should be contained in the curriculum? 

3. What methods should be used for the most economical and 
effective learning? 


F. J. Kelly, “Needed Research in Higher Education,” 
Studies in Education, pp. 74-84. Yearbook Number XV 
of the National Society of College Teachers of Education. 
Chicago: University of Chicago Press, 1926. 
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F. J. Kelly, The American Arts College, New York: The 
Macmillan Co., 1926. Pp. xiv + 198. 


Cost studies 

Student registration or enrolment 
Teaching load 

Miscellaneous statistical studies 
Experimental colleges 

Aims of higher education 

Learning problems 

Adaptation to different intelligence levels 
Measurement of the results of teaching 


ponon pony 


L. V. Koos, “Research Problems in Collegiate Education,” 
School and Society, XVII (February 17, 1923), 169-174. 


1. Functions appropriate to the last two years of the four-year 
college course 

2. Occupational destination of college graduates 

3. Differentiation of work to meet the needs of students of varying 
mental capacities 

4. Economic and social composition of the students of our colleges 

5. Extent of overlapping in high-school and college courses 

6. Development and use of achievement tests at the college level 

7. Methods of instruction at the college level 


E. A. Lee, “Research Problems in Training Vocational 
Teachers,” School and Society, XXIV (July 10, 1926), 
31-37. 


Selection of candidates for teaching 
Measurement of trade ability and trade testing 
Curriculum construction 

Evaluation of the performance of students 
Analysis of the trade teacher’s job 


nP 


A. R. Mead, “List of Possible Studies and Researches in 
Supervised Student-Teaching,” Educational Administra- 
tion and Supervision, XI (May, 1925), 355-358. 


1. What is the psychology of learning involved in acquiring 
ability to teach? 


2. What are the objectives of the various types of laboratory work 
in teacher-preparation? 

3. What are the curricula, facilities, and procedures needed to 
attain these objectives? 
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4. How to diagnose and evaluate the laboratory work of teacher- 
preparation 

s. Value of different plans for student-teaching, e.g., the Wis- 
consin and Cincinnati plans 

6. Proportion of teachers-in-service who have had student-teaching 

7. Relative efficiency of teachers-in-service who have had student- 
teaching as compared with teachers similarly situated who have not 
had student-teaching 

8. Establishment of norms for student-teaching done in one-room 
rural schools 

9. Standards for student-teaching of all types of teaching and 
supervision 

to, Student-teaching in foreign countries 

11. Historical antecedents and analogies for student-teaching 

12. Objectives, procedures, and evaluation of results in the ob- 
servation of teaching 

13. Survey of student-teaching in the various states 


A. R. Mead and Others, “Research in Supervised Student- 
Teaching and Allied Problems: Report of Research Com- 
mittee, 1926,” Educational Administration and Super- 
vision, XIL (May, 1926), 346-351. 


Investigations dealing with fundamental principles: 

1. Dealing with objectives of teacher preparation and attempting 
to answer the following questions: 

(a) What are the kinds of changes to be made in the teacher 
by his or her preparation? 

(b) To what degree should these be sought in pre-service 
preparation, under existing conditions? 

2. Dealing with the assumption that teachers learn to teach by 
the same laws of learning as they learn other things, że., dealing 
with the implications and applications of the psychology of learning 
to learning to teach 

3. Dealing with the establishment of standards for institutions and 
teacher-training activities, in order to apply principles of learning 
and attain desired objectives 


Continued series of investigations of existing theories, practices, 

material facilities, finance, staff, etc.: 

1. Collection of data about such work in Canada, Mexico, Central 
America, and South America 

2, Collection of such data in countries of Europe, India, China, 
Japan, Philippines, Asia, and Africa 

3. Experimentation to develop techniques adapted to research in 
this field 
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T. W. Merriam, “Needed Studies in Education in Religion 
at the College Level,” Religious Education, XXVI (Janu- 
ary, 1931), 32-37. 


Forty problems relating to: 

I. Courses in religion 

2. Religious values in other courses 

3. General problems of higher education to which religious 

education teachers may contribute 

4. Administrative relationships 

5. Professional college religious leadership 

6. Extracurricular provisions for religion 

7. Extracurricular activities in general 


Alonzo F, Myers, “An Evaluation of Research on Teacher 
Preparation,” Journal of Educational Research, XXVI 
(November, 1932), 161-174. 


1. What makes teachers effective? 

2. Further evaluation of the job-analysis technique 

3. How much and what kind of laboratory school experiences are 
essential in the professional preparation of teachers? 

(a) What activities should be included in the laboratory-school 
experiences of the student? 

(b) What are the respective contributions of observation and 
practice in the practical training of the teacher, and to what 
extent can the former safely be employed as a substitute 
for the latter? 

(c) Which is to be preferred, concentrated or extended periods 
of practice? 

(d) How much time should be devoted to observation and 
practice in curricula of two years, three years, four years, 
and five years in length for the preparation of elementary 
teachers and of secondary teachers? 

4. What is the best type of board of control for teacher-training 
institutions? 

5. Selective admissions to training programs 

6. Supply and demand in teaching 


Ruth Strang, “The Relation of the Dean of Women to 


Research,” Teachers College Record, XXXI (October 
1929), 44-49. 


1. Study “best-all-around, average, and problem” 
years and post-graduate life 


2. Trends in the number, causes, and kinds of disciplinary cases, 


girls for four 
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illnesses, and failures in academic work; and in personal problems, 
social activities, twenty-four hour schedules, and other phases of 
student life 


W. E. Uphaus, “Some Problem Areas in Higher Educa- 
tion,” Religious Education, XXVI (November, 1931), 
735-741. 


1. Spiritual inadequacy of compulsory chapel services 
2. What are the essential elements of a religious program? 

3. How to capture the latent energies of college youth for high 
spiritual endeavor 

4. What character assets do students bring with them to the 
campus? 

5. Point of departure of the religious education of a student 

6. What transformations in attitudes and ideals take place month 
by month and year by year? 

7. How do students invest their time, energy, and money? 

8. What curricular courses in religion are needed? 

9. What are the religious possibilities and effects of secular 
courses? 

to. How may they be brought out skilfully? 

11. What are the objectives of religious education? 

12. What are the best means of attaining them? 

13. Analysis of campus situations characterized by generous 
reconciliations and friendly codperation to reveal social and psy- 
chological factors involved 

14. Given certain campus conditions, what are the necessary ad- 
justments to shape a mutually helpful and creative student com- 
munity? 

15. Community control 

16. What techniques do local groups use in resolving their diffi- ` 
culties? 

17. What is the student's estimate of the relative value of the 
things he does? 

18. Does he experience contentment and satisfaction, or discourage- 
ment? 

19. Student-faculty relations 

20. Inter-racial relations 

21. Student government 

22. Honor system 


F. L. Whitney, The Junior College in America, p. 216. 
Colorado Teachers College Education Series, No. 5. 
Greeley, Colorado: Colorado State Teachers College, 


1928. 
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zr, To determine the place of this new educational unit (the junior 
college) in a state system of education 

2. To establish any possible contributions which the junior college, 
both public and private, may make in a local system of democratic 
education 


J. W. Withers, “Important Problems for Research in the 
Education of Teachers,” School and Society, XXXIII 
(May 9, 1931), 613-617. 


1. Are we making the most intelligent and effective use of re~ 
search results of the last twenty-five years in the preparation of 
teachers and the actual education of children? 

2. To what extent are these results wisely incorporated in the 
curricula and courses of study of present programs and practices in 
the education of teachers? 

3. What effect are they really having on supervision and instruc- 
tion? 

4. Proper recognition and rating of bachelor’s degrees of teachers 
colleges and normal schools by state departments of education and 
graduate schools 

5. Choice of the best source of high-school teachers: colleges of 
arts and sciences, teachers colleges, or fusions of the two 

6. The integration and articulation of teacher-training agencies 
and the cultural and professional aspects of the education of teachers 


The Articulation of the Units of American Education, 
Chapter XXVI. Seventh Yearbook of the Department 
of Superintendence. Washington: Department of Super- 
intendence, 1929. 


A. Problems of Articulation of Teacher-Training Institutions with 
Public School Units 
1. The selection of students 
2. Placement and follow-up 
3. Relation between pre-service and in-service training for 
teaching, administration, and supervision 
4. Certification for professional school service 
5. Curriculum studies in teacher training 
6. Training school facilities 
7. Articulation of high schools and teacher-training institutions 
8. Research service in state school systems 
g. Determination of faculty qualifications 
to. The arts college and teacher training 
tr. Legislation affecting administrative control of teacher training 
12, Locating teacher-training agencies within the state 
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13. Relation of probable tenure to the pre-service education of 
teachers 
14, State versus city teacher-training agencies 
B. Problems of Articulation within and among Teacher-Training Units 
1. Allocation of functions among the teacher-training agencies 
Optimum size of the teacher-training institution 
Methods of financing teacher training 
Methods of disbursement and accounting 
Determination of a usable unit cost in teacher training 
Social education 
Health service and health education 
Library service 
9. Physical plant and material equipment 
to. Faculty organization 
11. Transfer of credits 
12, Publications 


2a Anse p 


B. SECONDARY EDUCATION 


E. J. Ashbaugh, “Research Needed to Adjust High School 
Curricula to Educational Needs of the Individual,” 
Journal of Educational Research, XX (September, 1929), 
132-133. 


1. What constitutes a need? 

2. Determination of specific needs of individuals 

3. Differentiation between individual needs and general school 
aims 

4. Aims of the secondary school 


L. C. Bain, “Measures of Secondary-School Organiza- 
tion,” Educational Research Bulletin, XIV (November 
13, 1935), 201-207. 


Predicts that more consideration, as secondary education improves, 
will be given to a large number of variables in terms of pupils, parents, 
teachers, principals, buildings, curriculum, and economic, industrial, 
and social conditions, such as: 

x. Cultural background of pupils 

2. Attitude of parents toward secondary education 

3. Intelligence of pupils 

4. Economic level of the group served 

5. Probable kind of employment of schooling after secondary- 
school attendance terminates 

6. Social needs of the group servc& 

7. Attitude of group served toward society 
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8. Attitude of society toward group served 

9. Location, type, arrangement, and facilities of building 
to. Attitude of the teachers toward recent trends in education 
11. Educational philosophy of the principal 

12. Curricula 


T. H. Briggs, “Problems in Secondary Education That 
Need Solution,” School Life, XI (February, 1926), 116- 
118. 


The primary need is an acceptable philosophy which is compre- 
hensive, complete, consistent, convincing, and generally accepted, The 
history of secondary education has been neglected. Research, although 
it is in a formative stage, will provide the foundation by furnishing 
the facts. Scientific workers should choose subjects wisely, employ 
sound technique, report studies clearly, and popularize results. Based 
on reaction of forty-five writers, the number of problems proposed in 
the more popular fields are: 


1. Curriculum (48) 6. Tests (16) 

2. Methods (21) 7. Finance (9) 

3. Organization (19) 8. Prognosis (7) 

4. Teachers (19) 9. The Principal (6) 
5. Psychology (17) to. Supervision (6) 


T. H. Briggs, “If There Were Millions,” Teachers College 
Record, XXXV (May, 1934), 633-666. 


Indicates a program for education, proposing the following steps: 
Defining the good life 

2, Planning comprehensively for secondary education 

3. Determining the special functions of secondary education 

4. Learning from foreign schools 

5. Knowing the pupil population 

6. Defining education 
7 
8. 


H 


. Determining types of organization 
. Listing desirable activities 
9. Preparing teaching units 
to. Directing and codrdinating research 
11. Developing teaching units under ideal conditions, and under 
normal conditions 
12, Preparing curricula 
13. Measuring results 
14. Popularizing the program 
15, Training teachers of unusual promise, involving teacher-training 
institutions 
16. Devising a record system 
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17. Planning for guidance 
18. Providing for articulation 
19. Planning for buildings and equipment 


T. H. Briggs, “Research in Secondary Education,” Junior- 
Senior High School Clearing House, IX (December, 
1934), 198-205. 


What becomes of high school “drop-outs” and graduates? 
Social and physical effects of acceleration 
The results of programs varying in the number of subjects 
. The results of outside work and study on achievement in a 
normal-school program 
5. The effect of personal associations on academic school achieve- 
ment 
6. The persistence of high school learning after graduation 
7. Comparison of programs and achievement of students’ high- 
school and college careers 
8. Learning outside the school 
9. Plan of library classification of educational literature 
1o. Extent to which principals have utilized “scientifically proved” 
results A 
11. To what extent do principals earn their salaries? 
12. What do the department heads do? 
13. Distribution of administrative duties in many schools and in 
several types of organization 
14. The work of the state high-school inspector 
ts. The use of learning in other subjects in conventional curricula 
16. The effects of midyear graduation 
17. The later careers of “creative youth” 
18. Adapting adolescent education for the dull 
19. Delinquent youths in schools and courts 
20, What becomes of Protestant youth in New York? 
21. Reading on adolescent levels 
22. Education of the emotions 
23. The causes, degrees, and effects of emotionalized attitudes 
24. Extracurricular field 
25. The effects of extensive reading in foreign-language fields 
26. The effects of collateral reading in English, history, science, etc. 
27. Is literature ordinarily taught with regard to the principle of 
unity 
28, The results of teaching by different methods in various subjects 
and on the several intellectual-maturity levels 
20. How is the ability to systematize acquired knowledge related 
to different degrees of native intelligence? 
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30. To what extent does the systematization contribute to the re- 
tention, extension, and use of knowledge? 

31. Extent of retention after formal examinations 9 

32. What provisions have been made for integration of the in- 
dividual with larger social groups in the curricula? 

33. What provisions have been made in recent curricula for the 
integration of personality? 

34. How do intellectual interests begin and develop? 

35. To what extent has a guidance program been effective and 
useful? 


B. L. Johnson, The Secondary School Library, pp. 108-110. 
Office of Education Bulletin No. 17, 1932. National 
Survey of Secondary Education Monograph No. 17. 
Washington: Office of Education, 1933. 


1, Determination of adequate qualitative as well as quantitative 
standards for school library service 
2. Effect of newer methods of classroom teaching on use of the 
library 
3. Effect of library devices and activities on use of library materials 
and on the reading habits of pupils 
Relation of the library to the study hall 
Relation of the school library to the public library 
Selection of books 
Instruction in the use of books 
Relation of teachers to the library 
Training of librarians 
Discovery of additional library procedures used successfully 


DGD e Oy 


J. M. Lee, “Needed Developments in Measurement for 
Secondary Schools,” Education, LIII (March, 1933), 
424-427. 


Better tests 

. Training teachers in making and using tests 

Constructing new-type tests based on the local course of study 
point has had great emphasis in current literature. ] 

More prognosis or aptitude tests 

Tests which will measure outcomes other than factual elements 
More tests 

Better tests for vocational guidance 

More reliable norms 

Tests based on desired outcomes of instruction 
Comprehensive batteries with comparable scores 

Diagnostic reading tests 


4 
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C. W. Odell, “Educational Measurement in the Secondary 
School,” Journal of Educational Research, XXVI (Oc- 
tober, 1932), 81-89. 


1. Effect upon pupils’ achievements and attitudes of the various 
types of measuring instruments employed and of the various conditions 
under which they are used 

2. Effect of measurement procedures upon pupils, not only when 
the tests are employed and immediately thereafter, but at the end of 
the secondary school, through and at the end of the college program 
and thereafter 

3. Better prediction of success 

4. Critical study of present and future tests 

5. Construction of new measures—prognostic test, unit test 


W. C. Reavis, “Secondary Education as a Field for Re- 
search,” Phi Delta Kappan, XII (June, 1929), 21-24. 


1, The secondary-school curriculum, involving basic reorganization 
and improvement 

2. Individual or type activities in the extracurricular field 

3. Personnel administration, involving the school’s responsibility 
for adjustment, attacked by the case method and through studies of 
marks, ratings, records, and measurements, to appraise conditions as a 
basis for determining changes in policy 

4. School costs relating to unit costs of different subjects of in- 
struction, subject failures, extracurricular activities, service organiza- 
tions, and proportional distribution of costs among the different school 
functions 

5. Controlled classroom experimentation to solve such questions as 
the size of class most economical in pupil learning, relative merits of 
different methods of conducting laboratory work in science, efficiency 
of different classroom techniques, and the relative worth of visual aids 
and reading materials as supplements to assignments in textbooks 


P. M. Symonds, “Needed Research in the Field of Meas- 
urement in Secondary Education,” Journal of Educa- 
tional Research, XVI (September, 1927), 119-126. 


A list of 80 problems under seven headings: 

1. Characteristics of tests 

2. New tests 

3. Characteristics of mental growth of children of high-school age 
4. Use of tests in teaching 

5. Problems relating to marks, pupil placement, and program 

6. Problems relating to the measurement of efficiency 

7. Problems relating to prediction 
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“Looking Forward in American Secondary Education,” 
School Review, XLIII (February, 1935), 82-86. 


Urges investigation of: 

1. The abilities, interests, activities, and opportunities of youth 
from twelve years of age upward 

2, Optimum size of the secondary school 

3. Improvement of instructional programs in small high schools 

4. Improvement of the curriculum 

5. Utilization of community resources on a codperative basis for 
educational purposes 

6. Determination of how well school officers and teachers can be 
trained to give guidance to pupils 

7. Articulation of the senior high school and junior college 

8. A new basis of standards for secondary schools 


C. PRESCHOOL AND ELEMENTARY EDUCATION 


F. S. Breed, “Special Methods in the Elementary School— 
Problems and Methods of Research,” in “Methods and 
Techniques of Educational Research,” Review of Edu- 
cational Research, IV (February, 1934), 29-35. 


t. Selection of the Activities of the Curriculum—TIs social utility a 
valid basis for selecting the content of courses? 

2. Gradation of the Activities—What principle or principles should 
be used in grading the material of the curriculum? 

3. Organization of Units—How should curriculum content be or- 
ganized for instructional purposes? 

4. Direction of Learning 


(a) To what extent should individual as opposed to group in- 
struction be employed? 
(b) Should instruction be incidental or systematic? 
(c) What results are obtainable when pupils, courses, and 
methods—all three—are properly differentiated? 
(d) To what extent should specificity as against generality of 
learning determine the method of instruction? 
(e) Should instruction proceed through subjects or projects? 
(f) Is the work-book superior to the textbook system of instruc- 
tion? 
(g) Should instruction be based upon the principle of accelera- 
tion or on the principle of enrichment? 
5. Diagnosis and Treatment of Disability—What will be the con- 
tribution of diagnostic and remedial methods to elementary-school 
children? 
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6. Measurement of the Results of Instruction—How can improve- 
ment in the reliability and validity of standard tests be most ex- 
peditiously brought about? [This problem is discussed briefly in Chap- 
ter VII of the present volume. ] 


B. R. Buckingham, “The Public-School Teacher as a Re- 
search Worker,” Journal of Educational Research, XI 
(April, 1925), 235-243- 


Points out the benefit of research by teachers to the educational 
world, and to the teacher, with some problems needing investigation in 
arithmetic: 

1. Should addition and subtraction combinations be taught together 
or separately? 

2. To what extent may transfer of learning to higher decade addi- 
tion and subtraction be relied upon? 

3. What methods of teaching the addition combinations will best 
serve to prevent counting on the part of the pupils in getting the 
answer? 

4. Should the combinations be taught in regular order? 

5. If a broken order is used, to what extent is the difficulty of the 
various combinations a proper criterion for determining the order? 

6. To what extent, in determining the order of teaching, should 
combinations be kept apart which have a common element? 

7. What superiority, measured by results, is attained when pupils 
are given practice material evenly distributed over all the combina- 
tions? 

8, How should drill be distributed in point of time? 

9. How serious is an error made in giving a combination? 

1o. Myers pleads for imaginative problems set up with many words. 
Thorndike wants as few words as possible, as long as there is clear 
meaning. On what type of problem can children do best? 


Bess V. Cunningham and Ruth Connor, “Suggested Ex- 
periments in the Field of Pre-parental Education,” 
Teachers College Record, XXXIV (January, 1033), 285- 
301. 


This analysis of high-school pupils’ choice of home and family topics 

for inclusion in the curriculum suggests the following problems: 

x. Organization of certain aspects of pre-parental education with 
problems of vocational guidance and personal adjustment 

2. Further analysis of boys’ preferences for home-economics study 

3. Relation between child-care skills and child appreciation 

4. Experimental teaching unit, for eighth-grade girls, based on sug- 
gested interest in personal responsibility in the family 
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s, Teaching material for ninth-grade girls aimed at acquisition of 
child-care techniques for immediate use 

6. Material for same group aimed at development of appreciation 
of healthy childhood 

7. Relative importance of career or marriage 

8. Experimental teaching unit of social-science content concerning 
“home life in other lands” and “rules governing international trade” 


W. S. Gray, “Problems for Scientific Investigation in Ele- 
mentary Education,” Elementary School Journal, XXIII 
(May, 1923), 654-663. Of course, much evidence has 
accumulated since this analysis was made. 


1. How to reorganize the eight-year school into a six-year unit 

2. Functional codrdination of different administrative units 

3. Analysis of elementary-school subjects to determine the most 
appropriate period in which to emphasize each important phase of 
subject-matter 

4. Criteria for classification of pupils at various levels 

5. Determine the advantages and disadvantages of departmental 
and traditional internal organization of schools 

6. Relation of number of pupils per class to the efficiency of in- 
struction 

7. Costs of instruction per pupil 

8. Pupil experiences which form the core of the elementary-school 
curriculum 

9. Extent to which local conditions should influence the curriculum 

to. Courses of study meeting the needs of pupils classified by 
ability 

11, Analyses of the relationships between bodies of subject-matter, 
and experiments to determine the growth of pupils in knowledge, in- 
tellectual power, and skill under different principles of organization and 
presentation of subject-matter 

12. Development and refinement of general and specific tests 


W. H. Hughes, “Progressive Education and the Need for 
Research,” Educational Method, XI (January, 1932), 
197-201. 


Determine what is really progressive 

Variety of goals toward which education should strive 
Ultimate goals and intermediate objectives of education 
Tenable position on freedom of the learner 4 
Lifelikeness of school situations 

Qualities of personality requisite for vocational success, and edu- 
„cational situations productive of the same in students 
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7. Techniques for measurement of non-informational accomplish- 
ment 

8. Discrepancies between the real and supposed in education 

9. Post-rationalistic nature of innumerable and ambitious claims 
concerning new departures in education 

1o. Incompatibility of educational practices 

11, Inarticulation between home and school, school and workaday 
world, and units in the school system 

12. Principles of personnel management 

13. Personal qualities of successful teachers 

14. Content of the public mind 

15. Informing the community concerning its schools 


C. H. Judd, “Research in Elementary Education,” Journal 
of Educational Psychology, XVII (April, 1926), 217-225. 


1. Investigations into the typical characteristics of pupils of dif- 
ferent ages 

2, A study of the social institutions to which we introduce pupils 
of the school 

3. Problems in arithmetic content and method 

4. Reconstruction of the first six grades, following the loss of the 
seventh and eighth grades 

5. The field of school administration 

6. A study of the taxing system 

7. The fundamental principles of human relations in the institution, 
involving personnel management, management of teachers, and manage- 
ment of pupils 

8. The preparation of textbooks and other mechanical aids to in- 
struction 

9. The operations of the classroom 


Elizabeth S. Moore, “Can a Teacher of Young Children 
Carry on Research?” Childhood Education, VI (January 


1930), 197-200. 


1. What do young children gain from playing in a social group? 

2. How large may these groups be at progressive age levels? 

3. The relation in young children of social adjustment and self- 
direction 

4. Rôle of the teacher in fostering assertion and conformance 

5. What play materials have most value for children of advancing 
age levels? 

6. Values of playground apparatus? X 

7. Rôle of the teacher in introducing the child to various uses of 


materials 
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8. In what kind of environment do children express themselves 
freely in language, music, design, and construction? 

9. How may the teacher foster the creative interest? 

io. How foster appreciation? 

1r. Procedures of the adult when the child is eating 

12. What adult methods enable a child to relax? 

13. How acquire motor control and poise? 

14. How may the parent be educated? 

15. What data does research offer on the treatment of shyness, 
temper, fear, over-demonstrativeness, lack of perseverance, and un- 
kindness? 


Ruth E. Pearson, “Preschool Personality Research,” Ameri- 
can Journal of Sociology, XXXVII (January, 1931), 584- 
595. 


1. Studies of play of preschool child in neighborhood groups 

2. Variations in group behavior among children of the same age 
and analyses of the processes back of the behavior 

3. What a given person does and feels in response to others in all 
group situations which he habitually meets 

4. Results when the social codes and control techniques in two 


or more of the child’s earliest groups are mutually complementary, 
and when they are conflicting 


D. THE CURRICULUM 


T. H. Briggs, Curriculum Problems. New York: The Mac- 
millan Co., 1926, Pp. xiv + 138. 


1. What are the desired ends of education? 

2. What is “the good life”? 

3. To what extent is it desired that education shall modify the 
character and actions of future citizens? 

4. For which of the approved ends is the public school responsible? 

5. What details of subject-matter can contribute to the desired 
ends of education? 

6. What basis should be used for the selection of details of subject- 
matter? 

7. What should be the content of each course of study? 

8. What should be the plan of organization of the approved details 
of subject-matter? 
9. What are the responsibilities of each unit of the school organi- 
zation? 

10. What psychology has a bearing on the curriculum? 
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1r. What should be the distribution of the details of a course of 
study? 

12. What is the relative importance of each course of study? 

13. What is the best distribution of the time allotted to a course of 
study? 

14. What is required by law or by other outside agencies? 

1s. How long should education be continued at public expense? 

16. What is the optimum length of the school day and year? 

17. What is the optimum number of subjects to be carried simul- 
taneously by a pupil? 

18. Which units of instruction should be required and which 
elective? 

19. What lesson preparation at home can be secured from pupils? 

20, What degree of mastery is demanded? 

21. What are the possibilities of curriculum advance in the school 
community? 

22, What are the possibilities of curriculum advance with the 
teachers of the school? 

23. What are the possibilities of curriculum advance with existing 
buildings and equipment? 

24. What are the possibilities with available textbooks? 

25. What are the limitations imposed by administrative procedure? 

26. What are the characteristics and the most probable future needs 
of pupils? 


Harl R. Douglass, “Scientific Investigation of Instructional 
Problems,” Journal of Educational Research, XXIX 
(October, 1935), 130-138. 


Deals with field research involving the curriculum, methods of 
teaching, and tests 
t. The selection and organization of materials of instruction 
2. Techniques of assignment, exposition, and supervision of study 
3. The measurement of learning and growth 
4. Relationship between attendance and scholarship 
5. The use of work-books instead of note-books, or in addition to 
textbooks 
6. The effects upon achievement of different lengths of class 
periods 
7. The difference in time of day for recitation periods, of number 
of class meetings a week, or of different lengths of school term 
8. Reliability and validity of classroom tests 
9. Item analysis of tests revealing the relative reliability, validity, 
and descriminative properties of individual test items and types of 
items 
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Henry Harap, “Next Steps in Curriculum-making,” Ele- 
mentary School Journal, XXXI (September, 1930), 


16-24. 

x. Assembling the results of studies of objectives into a coherent 
whole 

2. Organization of the unit of work 

3. Form of courses of study 

4. Use of results of learning studies 

5. Adapting instruction to individual differences 

6. Grade placement 

7. Time allotment 

8. Evaluation of courses of study 


A. K. Loomis, “Needed Research in the Curriculum,” Jour- 
nal oj Educational Research, XXIX (October, 1935), 
126-130. 


Involving the historical method: 

1. Well-kept records of actual teaching procedures 

2. Case studies of pedagogical growth 
Involving controlled experimentation: 

3. Does the logical organization of subject-matter provide the 
most effective sequence of learning experiences? 

4. If the activity movement abandons all attempts to organize the 
sequence of learning experience according to logical demands of sub- 
ject-matter, what wiil be the results of this movement? 

(a) In the junior high school the wide use of fusion courses 
presents the issue in a slightly different form. 

(b) In the senior high school the use of problem courses raises 
the issue. 


Shelton Phelps, “Curriculum Construction,” Peabody Jour- 
nal of Education, V (January, 1928), 223-229. 


I. Principles of curriculum construction 

2. Skills and phases of knowledge produced by present curricula 

3. Unit tasks of each of the tasks for which public education 
strives to train 

4. Amount of variation produced by different studies of the same 
problem, using the committee-conference method of curriculum study 

5. Test experimentally all principles of curriculum construction 

6. Best research method for curriculum study 

7. Parts of various research methods best for a synthetic method 
of curriculum study 

8. Checking results 
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Ethel I. Salisbury, “Needed Research in Course of Study 
Making,” Journal of Educational Method, IV (June, 
1925), 410-416. 


1. Guarantees of efficiency in school subjects other than time al- 
fotments 

2. Children’s speaking vocabulary 

3. Moral growth 

4. Procedures in education that will produce better results in 
morals 

5. Amount of moral growth accomplished in connection with ac- 
tivities 

6. Universal experiences of the ten-year-old child, lending oppor- 
tunity for honesty or dishonesty 

7. Best type of school for moral growth 

8. Mental age at which certain skills can be most economically 
learned 

9. Basic experiences prerequisite to reading ability 

ro, Construction of materials related to basic experience, vocabu- 
lary, and the like of the intended age level 

tr. Necessity of nature study being related to reading 

12, To what extent has research affected practice? 

13. Test: (a) tendency to use large, loosely organized projects to 
give meaning to skills, (b) the tendency to analyze materials and then 
to arrange these in the order of their difficulty 

14. Results of current educational systems and devices 

15. Efficiency of the departmental program 


A. S. Taylor, “Curricular Research Is Urgently Needed,” 
Junior College Journal, III (February, 1933), 246-248. 


1. Curricular aims should be obtained and evaluated 

2. Do the aims meet community needs and sentiment or demands 
of remote agencies, e.g, state universities? 

3. Are the aims rooted in national or narrowly localistic and super- 
ficial educational philosophy? 

4. Ascertainment of curricular offerings in public and private 
schools $ 

5. Are curricular offerings governed by dictates of the community, 
higher institutions, or tradition? 

6. Are enrolments in non-college preparatory or terminal cur- 
ricula increasing or decreasing? 

7. Differentiation in curricula designed for two groups in the lead- 
ing junior colleges—students training for upper-division schools of 
arts and sciences and those training for professional schools 
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8. Analysis of course offerings to determine realization of aims, 
library facilities, and supplementary material available A i 

g. Classification, indexing, and annotation of curricular experi- 
ments 


Edward L. Thorndike, “Curriculum Research,” School and 
Society, XXVIII (November 10, 1928), 569-576. 


1. What is to be learned? 
2. Uses of various elements of the curriculum 
3. Concerning learners 


C. W. Washburne, “Needed Research in Curriculum Build- 


ing,” Journal of Educational Research, XXI (March, 
1930), 221-223. 


Method in curriculum-making 

Provision for individual differences 

Means of socialization 

Use of children’s natural and specialized interests 
Learning through development rather than acquisition 
Integration of social and individual life 


ORE E Ey 


H. E. Wheeler, “Suggestions for Research on the Typog- 


raphy of School Textbooks,” Elementary School Journal, 
XXIX (September, 1928), 27-31. 


1. The point system, involving experiments on type size 

2. Study of a few general classes of type-faces, involving styles 
generally used in books for adults, style in which the lines of the let- 
ters are of equal width (similar to the Copperplate Gothic), and the 
“display” faces 

3. The space between lines of type, called leading 

4. Line length considered in relation to size of type 

5. An adequate measure of the quality of type, in terms of being 
horizontally condensed or expanded 

6. Margins and paragraph indention 

7. The effect of “natural” tinted paper on the eyes of pupils, and 
from the standpoint of hygiene the degree of opacity that is desirable 

8. Use of hand-printed materials, when the size and shape of let- 
ters have never been specified for any given distance 

9. Effect of soiled pages on the child 


G. M. Whipple, “Needed Investigations in the Field of the 
Textbook,” Elementary School Journal, XXXV (April, 
1935), 575-582. 
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1. Is the textbook an unnecessary pedagogical adjunct? 

2. Is it advantageous to replace a single set of one textbook with 
several sets of different textbooks? 

3. What is the extent and the nature of the demand for master 
copies of material to be reproduced by duplicatory devices? 

4. What are the merits—quality of material and cost per word 
being considered—of the very cheap, ten-cent-store type of books as 
compared with standard textbooks? 

5. Why are school administrators averse to buying elementary- 
school books that deviate from the format of the typical textbook? 

6. What is the validity, if any, of the increasing refusal of the 
schools to buy textbooks bearing a copyright date more than a few 
years old? 

7, In what specific respects are textbooks too difficult for the 
learner? If vocabulary and style are primary considerations, how are 
the difficulties these present best determined and best obviated? 

8. Can the tentative conclusions reached in the investigations of 
certain Boston publishers regarding optimal sizes of type and leading 
in Grades I and II be confirmed in higher grades? 

o. What is the pedagogical value of illustrations, especially of 
colored illustrations, in school textbooks? 

to, What do the schools really want for indexes in textbooks? 


E. SCHOOL-SUBJECT FIELDS 


R. R. Aurner, “The Status of Research in the Field of 
Business English,” Balance Sheet, XV (November, 1933)5 
III-116, 144. 


t. Definition and clarification of problems of prime importance 
2. Construction of standardized objective tests for evaluation of 
teaching methods, effectiveness, performance, and student achievement 


W. L. Beauchamp, Instruction in Science, Chapter VHI. 
Office of Education Bulletin No. 17, 1932. National Sur- 
vey of Secondary Education Monograph No. 22. Wash- 
ington: Office of Education, 1933. 


The size of a learning unit 

The use of several textbooks 

The committee method 

The topical method 

The use of scrapbooks, posters, and related materials 
The use of guide sheets 

The use of the pupil work-book 

The unit method of instruction 

Initiation of new work 


PEI AMRe DH 
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10. The use of the project metho 

tz. Lecture demonstration versus individual pupil experimentation 
12, Methods of demonstration 

13. Methods of recording laboratory exercises 

14. The use of detailed or general laboratory directions 

15. The use of visual aids 

16, Training in scientific thinking 

17. Testing the results of instruction 

18. Types of pupil activities 


G. H. Betts, N: L. Tibbetts, Blanche Carrier, and Jessie A. 
Charters, “Four Areas of Experimentation,” Religious 
Education, XXIII (March, 1928), 229-239. 


A. Religious Curriculum 
1. Loose versus close grading 
2, Range and type of information resulting from a specified cur 
riculum, 
3. The curriculum conceived as types of experience for pupils 
4. Proper sources of curriculum content 
5. The function of worship in a religious curriculum 


B. Local Church 
A miscellaneous list of methods of improving a Sunday School 


C. Teaching Methods of the Week-day School of Religion 
6. Need and method of balance between discussion, worship, and 
some form of expression in a group of lessons on one theme 
7. Balance between idealistic and materialistic or realistic material 
8. Methods of questioning and discussion 
9. Pupil participation in the learning experience 


to. Principles governing inductive and deductive methods of lesson 
approach 


11. Motivation 
12, Adaptation of material to varying pupils 
13. Function of home work, report cards, and memory work 
14. The steps in creating a desire to learn 
D. The Family 


15. A plan of religious education in the home which will develop 
the religious experience without sanctioning outworn beliefs 


O. W. Caldwell, “Research and Elementary Science Teach- 
ing,” Science Education, XVIII (April, 1934), 65-67. 


x. Accurate measurement of results _ l 
2. Criteria for selection of curriculum content 
3. Grade placement of subject-matter 
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F. D. Curtis, “Some Contributions of Research to Prac- 
tices in Science Teaching,” Science Education, XVI 
(April, 1932), 266-273. 3 


1. The determination through various analyses of the desirable 
and practical outcomes of classroom and laboratory work, as the logi- 
cal and practical basis for investigations into the relative merits of a 
wide variety of teaching methods as means of securing these specific 
outcomes 

2, Analyses for the purpose of determining a wider variety of 
profitable classroom, laboratory, and extra-classroom activities, de- 
vices, and practices distinct from teaching methods, with objective de- 
termination of the relative values of these 

3. Construction of standardized group tests for measuring a large 
number of outcomes of instruction other than a knowledge of subject- 
matter 

4. Synthetic curricular researches aiming to combine and utilize 
in an objective way the findings of miscellaneous related curricular 
studies 

5. Further analyses of objectives looking toward the determination 
of the influences which science may exert in molding thought. Espe- 
cially desirable in this connection are studies for the purpose of 
further determining and refining the major generalizations of science 
with their accompanying scientific attitudes and the methods by which 
these may most effectively and most universally be taught 


C. A. Ellwood, “The Fundamental Research in Educationai 
Sociology,” Journal of Educational Sociology, VII 
(September, 1934), 4-11. 


1. Can public spirit, democracy, social intelligence, and social atti- 


tudes be taught in the schools? 
2. Through control of learning situations, can any desired type of 


society be produced? 
3. Can the experience of the past, the wisdom of age, and the les- 
sons of history be imparted to youth and future generations? 


Gertrude C. Ford, “Proposed Research by Business Educa- 
tion Associations,” Journal of Business Education, 1X 


(April, 1934), 21, 34- 


1. Year-books in the content subjects of general and vocational 


business education at the different levels } 
2. Year-books in the professional areas for general material and 


higher aspects of business education ; PSH 
3. Ascertain the status of business education within the states for 


guidance nnrnoses 
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V. A. C. Henmon, “Recent Developments in the Study of 
Modern Foreign Language Problems,” Modern Language 
Journal, XIX (December, 1934), 187-201. 


Seven fundamental questions, the solution of which rests upon ade- 
quate experimentation: 

1. Who should and who should not study foreign language? 

2. At what age should the study of a foreign language begin? 

3. How long should it be continued, to yield profitable and worth- 
while returns? 

4. What should be the specific objectives of a two-year course, 
since 85 per cent or more of pupils study a language for two years or 
less? 

5. What should the curriculum content of the two-year course be? 

6. What methods are most effective in realizing the objectives of 
the two-year course? 

7. What standards of achievement may be expected, and how 
should they be expressed? 


L. T. Hopkins, “The Function of Research in Public School 
Home Economics,” Journal of Home Economics, XXII 
(May, 1930), 358-364. 


Defining aims 
Determining content and method 


Discovering outcomes 
Training of teachers 


Benny 


G. C. Jensen, “Problems of Commercial Education,” Bal- 
ance Sheet, XI (February, 1930), 174, 178. 


1. Relation of commercial community needs to commercial school 
programs 


2, Placing of proper persons in jobs by means of high-school 
recommendations of fit students 


C. H. McCloy, “Needed Research in the Field of Pro- 
gram in Physical Education,” Research Quarterly of the 
American Physical Education Association, II (October, 
1931), 78-84. 


t. What objectives of physical education are significant to the 
pupil? 

2. The individual 

3. Criteria for instructional material 

4. Application of principles of mechanics to progression in physical 
activities 

5. Psychology of fundamental teaching procedure 

6. Techniques of measurement, classification, and marking 
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Research in Unit Activities 


7. The objectives of each activity 

8. Pupil attitude toward the activity 
9. Content of the activity 

ro. Organization of activity materials 
1r. Teaching procedure 

12. Pupil activity 

13. Measurement of results 


Research in Organization 


14. Studies of programs for varying individual needs 
15. Group motivation 


C. H. McCloy, “Research in Athletics for Girls and 
Women,” Research Quarterly of the American Physical 
Education Association, III (October, 1932), 101-104. 


1. Is any given athletic exercise or competition harmful to the 
health when conducted in a stated manner? 
Same for intramural and interschool competition 
Causes of harm in harmful exercises and competition 
Determination of optimum strength for girls and women 
Educational and emotional effects of athletic competition 
Organization and teaching of activities having leisure-time 
carry-over values 

7. Classification of pupils in terms of physical and emotional 
qualities 

8. What are the fundamental skills of the educationally important 
activities? 

9. How can these skills be best taught? 

10. Studies in dynamic body mechanics 

11. Scientific basis of scoring systems 

12. Measurement of “athletic intelligence” 

13. Measurement of present and potential physical status 


J. W. Martin, “What Business Research Can I Do This 
Summer?” Journal of Business Education, IV (July, 


1930), 17, 24. 


OLY» 


Historical 


1. Development of some particular commercial or manufacturing 


enterprise 
2. History of a trade area ; R 
3. History of a navigation or other transportation enterprise 


4. History of a codperative society 
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History of a local chamber of commerce 
History of drug-stores 


Nes 


Management 


An analysis of the accounting system of some company 
An analysis of the inventory system of some company 
9. Family budget studies 


Smt 


E. G. Payne, “Research Problems and Trends in Educa- 
tional Sociology,” Journal of Educational Research, XXV 
(April-May, 1932), 239-252. 


1. The relation of education to the problem of community health 

2, The relation of standardized curricula, school activities, and 
inethods of instruction to variations in local communities 

3. The education of cultural and racial minorities 

4. The school use of community resources 
The use by the community of school resources 
The control of education by social forces in the community 
The use of leisure time in relation to the school 
The relation of the school to crime prevention 

9. The sociological significance of various curricular changes in 
the recent past 

to. The sociological determination of curricular requirements by 
special studies of community needs 


px on 


Anne E, Pierce and R. S. Hilpert, Instruction in Music and 
Art, p. 65. Office of Education Bulletin No. 17, 1932. 
National Survey of Secondary Education Monograph 
No. 25. Washington: Office of Education, 1933. 


1. (a) A comprehensive study of the art curriculum as it exists 
in secondary schools; (b) research on the extent to which knowledge 
and appreciation of art function in adult life; (c) the development of 
a proposed curriculum based on this research (A beginning toward this 
was made in Minneapolis in 1932) 

2. A study of the influence of training on ability in art by com- 
parison of paired groups of those who do not take art with those who 
take it in the high school (This study would also include a comparison 
of different methods of instruction.) 

3. A study in methods to determine how to conserve self-expres- 
sion and creativeness typical of the elementary-school child and still 
gain skill in technical values of which pupils in secondary schools fee) 
need 

4. Research in the psychology of self-expression, imagination, and 
creativeness, including analysis of these experiences, leading toward 
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the development of tests for their discovery in pupils, and experiments 
to determine methods to develop these qualities 


Dora V. Smith, “Diagnosis of Difficulties in English,” Edu- 
cational Diagnosis, pp. 265-267. Thirty-fourth Yearbook 
of the National Society for the Study of Education. 
Bloomington, Tl.: Public School Publishing Co., 1935. 


1. There is urgent need of agreement upon the objectives of the 
teaching of English. Once determined, they must be stated with un- 
mistakable clarity and broken into specific elements capable of meas- 
urement or evaluation. What, for instance, are the skills involved in 
the “ability to marshal evidence or facts concerning a problem under 
consideration”? What is implied in the term, adequate vocabulary, or 
interesting style, so far as speaking and writing are concerned? [There 
has been a gap between broad aims and specific learning activities. ] 

2, Studies of the relationship between such factors as health or 
social adjustment and success or failure in English are all but non- 
existent. 

3. Investigation is needed of the relationship between enriched 
classroom environment and the development of such powers of ex- 
pression as adequate vocabulary and wealth of ideas. The sources of 
stimulation of creative writing are, for instance, practically unknown 

4. Experimental evidence is as yet lacking to prove the superiority 
of concrete, purposeful situations for teaching writing and speaking 
over artificial exercises and drills. 

5. With one or two possible exceptions, there are no diagnostic 
tests in English to-day that actually diagnose pupil difficulties. Studies 
are needed of (a) variation in performance due to the nature of the 
test used, (b) adequate mapping of the whole area of difficulty, (c) 
frequency of appearance necessary to adequate testing of an element, 
and (d) approximation in the test of actual language situations in 
speaking and writing. 

6. Process analysis akin to that in arithmetic must be developed in 
the field of English if pupil difficulties are to be obviated by preventive 
teaching. What factors, for instance, are involved in the single process 
of learning to use the apostrophe to show possession or to indicate 
connection or relationship? 

7. Instruments for analyzing power in composition are yet to be 
devised. What, for example, do we mean by forceful style, by concrete 
diction, by a wealth of ideas, and how can they be measured? What 
are the components of ability to organize ideas, and how can profi- 
ciency in it be tested? 

8. Measures for oral composition approaching in objectivity those 
for written composition are necessary to the evaluation of methods of 
teaching in that field. Standards for articulation, rate of speaking, 
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variety of tone, and other elements of oral technique are unknown to 
the average teacher. 

9. Scales or other instruments are awaited for the evaluation of 
results in certain functional centers of expression, such as the making 
of announcements, conversations, and creative writing. Determination 
of pupil progress in the writing of verse is impossible without stand- 
ards of judgment more objective than those available at the present 
time. 

10. Studies in the relative learning difficulty of items in English are 
almost non-existent. 

tr. Records must be made in the near future of the amount of 
practice in the actual expression of ideas the average pupil has in the 
classroom, 

12. Best methods of eliminating errors in the mechanics of English 
have been the subject of a few investigations. The importance of oral 
emphasis in drill and the value of grammatical analysis as an aid in 
the correction of errors are problems demanding immediate attention. 

13. Best methods of improving pupil control of sentence structure 
are urgently in need of investigation, in view of the difficulty inherent 
in mastery of the sentence. 

14. The Iowa technique for recording oral speech in the classroom 
reopens for investigation the whole field of pupil errors in English. It 
likewise makes possible a new and constructive attack upon old prob- 
lems—a study, for instance, of the gradual development of pupil con- 
trol over sentence structure—and provides limitless possibilities for the 
study of pupil powers of organization and the coherent development of 
ideas. [This technique and the appropriate equipment were mentioned 
briefly in Chapter VIII.] 

15. The problem of what is correct English usage has been little 
more than opened up by Leonard’s studies in that field, Analysis of the 
actual writing and speaking of adults is still ahead. So also is the de- 
termination of the relative cruciality of errors. 

16. Investigation of methods and content suited to the needs of 
pupils of varying levels of ability is of major importance in English, 
as it is in other subjects of study. 

17. Grade placement of materials on the basis of relative difficulty 
of items and pupil use and progress in mastery of them at various 
levels of instruction are matters for earnest inquiry at the moment. 

18. Problems of remedial instruction needing investigation include 
such factors as the relative value of motivated versus unmotivated 
remedial teaching, the relative effectiveness of individualized versus 
group instruction, and the extent to which pupils may be taught to 
assume responsibility for the diagnosis and remediation of their own 
errors. 

19. Objective recording of the study techniques of pupils who are 
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successful and unsuccessful in various phases of English would do 
much to facilitate the program of remedial instruction. 

20. Finally, check-lists and other devices for the objective record- 
ing of pupil attitudes and appreciations as revealed in concrete class- 
room situations are fundamentally important to the evaluation of cer- 
tain of the most vital outcomes of the teaching of English. 


Dora V. Smith, Znstruction in English, p. 88. Office of 
Education Bulletin No. 17, 1932. National Survey of 
Secondary Education Monograph No. 20. Washington: 
Offce of Education, 1933. 


1. What should be the relationship of high-school English to the 
general objectives of secondary education? 

2. How far does the present program in composition and in litera- 
ture meet the present or the future needs of adolescent boys or girls? 

3. To what objectives other than mere correctness should the com- 
position course contribute? 

4. To what extent does the present program in English grammar 
influence speech and writing? What is its relative importance in an 
overcrowded program of instruction? 

s. Granted that a pupil is of low intelligence and has but a year 
or two to remain in school, what program of English instruction will 
contribute most to his future welfare and efficiency? 

6. What is preparation for college; that is, what are the actual 
demands of higher institutions? To what extent should they dominate 
secondary-school practice? 

_7. Is there a common body of literary material with which all 
pupils should be familiar? 

8. What are minimum essentials and on what bases should they 
be selected? 

9. Granted that the major objectives of the teaching of literature 
are breadth of experience and interests and a habit of lifelong associa- 
tion with good books, what literature content and what methods of 
classroom instruction are best calculated to achieve these ends? 


DeForest Stull, “Needed Studies in the Field of Educational 
Geography,” Teachers College Record, XXXIII (Febru- 
ary, 1932), 416-421. 
1. What content for elementary schools is necessary for the realiza- 
tion of the citizenship aim? 


2. What are the fundamental principles of geography? 
3. What should be the grade placement of curriculum materials in 


geography? 
4. What are the learning difficulties involved in the study of geog- 


raphy in the various grades? 
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P. M. Symonds, “Needed Research in the Teaching of 
English,” English Journal, College Edition, XXII (June, 
1933), 447-456. 


1. List of the most frequent usages and errors 

2. Identify and teach those important cues and guides used in 
learning, writing, and speaking habits 

3. How are illustrations and models most effectively used? 

4. Can we teach the application of grammatical and rhetorical 
rules without having the child learn them as such? 
. Improvement of reading habits of high-school pupils 
6. Methods of instilling appreciation of literature 
7. What factors are related to interest in literature? 
8. Means of, and factors in, changing interest 


n 


Ruth Strang, “Health and Physical Education,” “Special 
Methods and Psychology of the Elementary-School Sub- 
jects” number, Review of Educational Research, V 
(February, 1935), 53. 


r. Continuous critical descriptive records of the development of 
programs of health and physical education in selected schools 

2. Continuous study of pupils as suggested above 

3. Intensive studies of relationships among factors such as health 
and scholarship, health and conduct, health habits and health knowl- 
edge, etc. 

4. Experimental situations in which the outcomes of certain 
changes in procedures are recorded 3 

5. Controlled classroom experiments in which the results of dif- 
ferent methods and materials are compared 

6. Controlled analytical laboratory experiments which furnish ac- 
curate facts on specific questions and suggest leads for ‘classroom pro- 
cedures 


J. F. Williams and C. H. McCloy, “Problems for Further 
Study,” Research Quarterly of the American Physical 
Education Association, II (March, 1931), 181-189. 


What is the aim of physical education in college? 
Objectives 
Realization of these objectives 
How important is exercise in the promotion of health? 
. Does exercise stimulate the growing child to increased growth 
and development? 
6. Posture 
7. What price athletics? 


YR eS om 
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8. In the professional training curriculum, what are the real needs 
as opposed to the merely traditional ones? 

9. Curriculum 

to. State certification as related to the professional curriculum 

11, Objectives of the teacher 

12, Interests and attitudes of the pupils 

13. Criteria for teaching material 

14. Best content of teaching material for attaining the aims 

15. Unit activity objectives 

16. Pupil attitude toward given activities 

17. Teaching procedure 

18. What do we expect the student to do in a given educational 
situation? 

19. Measurement of results 

20. Effects of injuries to varsity and non-varsity athletes 

21. Educational benefits of controlled athletics 


The Development of the High-School Curriculum, Part II. 
Sixth Yearbook of the Department of Superintendence. 
Washington: Department of Superintendence, 1928. The 
following analysis is adapted from F. L. Whitney, Meth- 
ods in Educational Research, PP. 47-50- New York: D. 
Appleton and Co., 1931. 


Commercial Studies 


1. The office and store occupations most frequently filled in the 
community by drop-outs and graduates from the school in which the 
course is to be used 

2. The opportunities for employment in offices and stores open to 
beginners from the school with such amounts of general and special 
training as can be given in the high school 

3. The positions to which beginners are promoted within a reason- 
able time after entering upon employment 

4. The duties, traits, and responsibilities required in each of the 
beginning and promotional positions most frequently filled by drop- 
outs and graduates, with special reference to the details required by 
local employers and determined by the particular nature-of the busi- 
ness and commercial life of the community 

5. The sequence of subjects that will best prepare both drop-outs 
and graduates for the positions open to them at the time they leave 
school 

6. The subject content and most effective classroom procedures in 
terms of learning activities and related traits that will function most 
effectively in giving abilities and appreciations needed in the charac- 
teristic beginning and promotional positions in the community 
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English 


1. Nearly all previous “errors” studies should be repeated 

2. The problem of how pupils organize what they have to say 
should be investigated 

3. Grade placement should be examined 

4. The measurement of achievement in composition should be at- 
tacked 

5. How far school instruction carries over into writing and speech 
should be determined 

6. Material and practice tests for work reading should be experi- 
mentally constructed 


Home Economics 


I. What to teach—what skills, information, appreciations, and at- 
titudes should be developed in girls who are living in parent homes but 
soon will be wage earners and later home-makers? 

2. How to select—values in curriculum materials, for case groups? 

3. How to arrange the materials—means and methods for unifying 
home economics? 

4. How to measure—means of measuring the functioning of home 
economics? 

5. How to relate—an understanding of the part of home economics 
in the education of both boys and girls? 


Mathematics 


1. Do we need curricula differentiated on the basis of intelligence 
levels? Precisely what things mathematical can pupils having low in- 
telligence do with a reasonable degree of success? [Ability grouping 
was discussed in 1936 by the National Society for the Study of Edu- 
cation. ] 

2. What is commercial arithmetic? Who needs it? Who can pursue 
it successfully? 

3. What do pupils taking the industrial-arts curriculum need that 
is to be found in the newer courses for the junior high school? 

4. What is the relative effectiveness of the different types of prac- 
tice materials used to establish skills in the junior high schools? 

5. What are satisfactory degrees of mastery for the fundamentals 
of arithmetic? 

6. If the present low level of mastery of the fundamentals of 
arithmetic manifested in the senior high school is not sufficient, how 
shall the courses of the senior high school be reorganized to build up 
the skills? 

7. Do pupils who study the newer courses in the junior high- 
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school succeed better in a course of demonstrative geometry of the 
senior high school than do pupils trained in conventional courses? 

8. What are the criteria for a desirable test of algebra? 

9. How effective are the various classroom techniques for teach- 
ing the skill of problem solving in algebra? 

to. How does the individualized plan for teaching the skills com- 
pare in effectiveness with the older group method? 

rz. To what extent can teachers with the usual heavy load do 
remedial work? 

12. How large must a school be before it can afford a “hospital” 
room in mathematics throughout the school day? 


Science 


1. Determination through various analyses of the desirable and 
practical outcomes of classroom and laboratory work, as the logical 
and practical basis for investigations into the relative merits of a wide 
variety of methods, including the individual laboratory and the demon- 
stration, as means of securing these specific outcomes 

2. Analyses for the purpose of determining a wider variety of 
profitable classroom, laboratory, and extra-classroom activities, devices, 
and practices distinct from teaching methods, with objective deter- 
minations of the relative values of these 

3. Construction of standardized group tests for measuring a large 
number of outcomes of instruction other than a knowledge of subject- 
matter 

4. Investigations to discover other perhaps more satisfactory and 
effective bases for grouping and sectioning pupils for class work and 
for experimentation in the teaching of science than intelligence or rate 
and comprehension of silent reading 

5. “Synthetic” curriculum researches aiming to combine and uti- 
lize in an objective way the findings of miscellaneous related curric- 
ulum studies 

6. Further analyses of objectives looking toward the determina 
‘ion of the influences that science may exert in molding thought 


Social Studies 


1. What peculiar purpose or objective should characterize the pro- 
ram of social studies in the secondary school? 

2. How can the program of social studies in the senior high school 
be articulated with similar programs in the elementary and junior high 
school? 

3. What end does the school contemplate, child needs or adult 
needs? Or both? 


820 EDUCATIONAL RESEARCH 


4. Should the merits and deficiencies of contemporary life be 
studied by senior high-school pupils, or should the courses be merely 
informative—descriptive sociology? Or should they be interpretative? 

5. How should the materials of the social studies be organized to 
insure the development of critical informed students of societal needs? 

6. To what extent can the particularized items—knowledges, skills, 
meanings, generalizations, appreciations, attitudes, and ideals—be de- 
termined objectively? 


F. EDUCATIONAL PSYCHOLOGY AND MEASUREMENT 


H. H. Anderson, “Research in Mental Hygiene,” Child- 
hood Education, VII (April, 1931), 423-427. 


1. What is the relation between childhood and adolescence with 
regard to personality development of school children? 

2. Does departmental teaching render unnecessary and impossible 
the consideration of personality development of school children? 

3. What is the cost to the mental health of the children of the 
school’s failure to develop personality? 

4. Are there more infractions (gum-chewing, unpermitted talking, 
shooting paper wads, writing notes) of this nature among junior high- 
school pupils than in lower grades? 

5. Are these problems more serious for adolescents? 

6. Reaction of the child to departmental organization 

7. Are whispering, unpermitted talking, and writing notes evi- 
dences of thwarting of integrative processes by teachers or by the 
system? 

8. Effects of “up-to-date” teaching methods on the mental hygiene 
of the child 


9. Pupil reactions, inferiority feelings and the like, to “special 
classes” 


Hugh Hartshorne, “Present Status of Research in Character 
Education,” Religious Education, XXV (June, 1930), 
551-554. 


1. Utilization of words as tools of character education 
2. Permanence of results of character education 
3. Transfer of character-education values 


B. R. Buckingham, “Fundamentalism in Research,” Journal 
of Educational Research, IX (April, 1924), 331-334. 


Believes there has been a tendency toward shallowness among re- 
search workers, who should apply sustained effort until fundamental 
results are secured. Mentions some problems for careful investigation: 

I. Transfer of training as a condition of learning 
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The learning types of children 

Forgetting and the distribution of recall 

Whole and part methods of learning 

The curve of work during a particular period 

The learning curve over successive periods 

Fatigue 

Motivation 

The meaning of the law of age of associations 

The rôle of the teacher in controlling these processes 


ONO COST ON re G2) BS 
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R. G. Foster, “Objective Methods of Sociological Research 
Generally Applicable to Child Development Studies,” 
Journal of Educational Sociology, IX (October, 1935), 
79-87. 


Covers needed research in the sociology of childhood: 

1. Sociological techniques and methods of child study 
Historical and comparative studies of the child in society 
Studies of the social attitudes and personality of children 
Social problems, pathology, and adjustments of childhood 
The social organization and institutions of childhood 
Population, ecological, and demographic studies of childhood 
Conflict and accommodation groups of childhood 


SPR oR 


V. A. C. Henmon, “Needed Research in the Field of Learn- 
ing,” Journal of Educational Research, XI (May, 1925), 
313-321. 


1. Preschool learning 
(a) Conditioned reflex 
(6) Instructive and emotional responses 
(c) Innate and acquired tendencies 


2. Psychology of learning in general 
(a) Learning curves of school activities 
(b) Continuity studies of learning 
(c) Distributed repetitions 
(d) Overlearning 
(e) Forgetting 
3. Psychology of learning school subjects under school conditions 


L. T. Hopkins, “Exploring New Areas of Measurement,” 
Curriculum Journal, VI (May 22, 1935), 22-25. 


Assumptions in a new type of measurement are: 
1. An intellectual heritage variable with the environment and 


maturity of the individual with emphasis upon experimental selection 
of meanings, generalizations, and techniques 
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2, An increasing control over learning by the learners 
3. A fragment of experience can be adequately measured only in 
relation to other fragments which form the emerging whole experience 
[This involves certain concepts of Gestalt psychology. ] 
(a) The purpose of the individual 
(b) The interests of the individual 
(c) Control over experience 
(d) Criticized values 
New forms of measurement which must be given greater weight are: 
1. Teacher's observations of the general behavior of pupils 
2. Diary accounts of the development of complete experiences 
3. Records of work proposed and accomplished by individuals 
4. Teacher’s records of instances of emotional instability or sta- 
bility of individuals 
5. Teacher’s records of lack of coöperation 
6. Teacher’s records of instances of lack of social stability or con- 
trol of social adjustments 
7. Performance of children in codperative group activities involv- 
ing the utilization in modified form in a new setting of the learnings 
acquired in some previous experience 


F. H. Lund, “Educational Research in the Field of Emo- 
tion,” Educational Record, XII (April, 1931), 138-143. 


1. Analyses of conditions surrounding those students whose 
achievement falls below the norm for their intelligence level 
2. Is interest, or drive, or incentive an acquired or native trait? 
3. Determination of the relative effectiveness of different forms 
of motivation in use in the school-room 
4. Understanding of the autonomic nervous system 
, 5. To what extent is education motivation and connection form- 
ing? 
Mapheus Smith, “Suggestions for Sociological Research in 
Child Development,” Journal of Educational Sociology, 
IX (October, 1935), 105-110. 


Preschool Period 


1. A study of the conditioning factors in home life 

2. Amount and quality of differences in development of preschool 
children reared in orphanages and at home 

3. Differences in development of preschool children of different 
social classes 

4. A study of the requests of preschool children in middle-class 
families—what they ask for or ask to be allowed to do; correlation of 
the requests with the cultural exposure of the child, to determine to 
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what extent requests of young children are contingent on cultural in- 
fluences, and to what extent on purely original and individual factors 
[This area is shared by sociology and psychology.] 

s. A study of disobedience of young children, in order to deter- 
mine at what points resistance to patterning of behavior appears, and 
under what conditions 

6. Relation of nursery-school experience to development of social 
traits in children 

7. Study of social processes by which undefined organic processes 
of the newborn infant become human nature 


Adolescence 


1. Differences in sophistication of children of different class and 
community backgrounds 
2, The effect of puberty upon social relationships 
3. The effects of institutional changes upon the personalities of 
children of junior and senior high-school age 
4. A systematic study of social problems of adolescence, suggested 
under the following heads: 
(a) The study of the new worlds to which the adolescent must 
adjust 
(b) A study of the definitions and interpretations which are 
made of these worlds 
(c) A study of the manner in which adjustments are made 
(d) A study of specific adjustments to objects and situations, 
ie., institutional obligations, sex behavior, courtship, etc. 
(e) A study of the life pattern and personality traits built up 
as a result of these serial efforts at adjustment 


Entire Age Range of Child Development 


1. Comparisons of orphanage and non-orphanage children 

2. The effect of the order of birth upon personality 

3. The impact of the pecuniary resources and organization of the 
family and neighboring families on the maturing personality of the child 
(This may be broken up into more specific problems, such as the 
child’s maturing needs for money and the sources of these needs, fam- 
ily “sets” towards money, the process of organizing his outlook on his 
world in terms of his spending resources, and social stereotypes as to 
kinds of orthodox expenditure.) ( 

4. Effects of shifting of economic class lines in depression periods 
on personality of the children involved 

5. The effect upon the child of shifting relationships growing out 
of the father’s inability to support the family 
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6. Experiences with the fact and idea of death, and their subjec- 
tive and objective effects on children k 

7. The extent to which the expectancy of others, both children and 
adults, influences the interests and actions of children 

8. A series of investigations in different centers covering such 
points on the employment of youth as (a) the age at which, and 
characteristics of the position in which the child’s adjustment is 
greatest; and (b) the alternatives to employment which are most and 
least closely related to satisfactory adjustment 

9. Relation of social and emotional problems of children to the 
number and variety of social contacts they make with social agencies 
and institutions, with social groups, and with industrial and street life, 
both within and outside of their own communities 

to. Origin, nature, and development of social concepts in children 

11. Attitudes of children towards certain selected laws 

12. Further studies of children’s imaginary companions 

13. Further comparative studies of the development of identical 
twins separated before the age of six months and placed in foster 
family homes 

14. Further study of the effects of foster home placement on per- 
sonality 

15. A comparative study of the rearing and development of chil- 
dren of deaf-mute parentage and of children of normal hearing parents 
of the same social class and local community 

16. The comparative speed of development in social learning and 
sophistication of white and Negro children in order to learn more 
about race differentials in development 

17. Further studies of conditions under which delinquent and 
criminal tendencies, moral ideas, moral habits, and strength of char- 
acter develop 

18. A study of antisocial traits of juvenile delinquents in relation 
to family and neighborhood backgrounds 

19. Studies of the development of tendencies to Icad, to follow, to 
originate, and to imitate 

20. Development of in-group consciousness in children 

21. Observational and experimental studies of the effect of face-to- 
face, sympathetic interaction upon the development of personality 

22, Development of tendencies to play group rôles 


23. Further studies of sex differences at various ages, especially 


through adolescence, with a view of determining the effect of the réle 
of each sex in determining sex differences 

24. Studies of race differences in the production of persons with 
ability and personality for outstanding leadership, including studies 
of Negro children of superior intelligence, after the manner of the 
Stanford studies of gifted children 
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25. Methods of obtaining coöperation and socialization 

26. Methods of obtaining morale, rapport, and esprit de corps, and 
the methods of inculcating traditions 

27. Means of transmission of customs and mores to the younger 
generation 

28. The extent to which psychopathic tendencies are the result of 
experience and environmental factors 


P. M. Symonds, “Needed Research in Diagnosing Personal- 
ity and Conduct,” Journal of Educational Research, 
XXIV (October, 1931), 175-187. A list of 205 problems 
under 14 main headings. 


1. How reliable is a test, questionnaire, or other device for secur- 
ing information regarding personality and conduct? 

2, What conditions affect reliability? 

3. How valid is a test, questionnaire, or other device for securing 
information regarding personality and conduct? 

4. What is the effect of changing some elements of the situation 
in any one of the diagnostic techniques considered? What is the influ- 
ence of various factors on test results? 

5. What is the relation between various measures and estimates 
of personality, conduct, emotion, etc.? 

6. What are the possibilities for improving the various diagnostic 
techniques? 

7. What qualities are necessary for success in the investigation? 

8. The development of new tests, questionnaires, and other de- 
vices 

g. What is the distribution within a group of amounts of a trait, 
interest, attitude, adjustment, etc.? 

10. The problem of framing items, questions, etc. 

11. The problem of selecting items 

12, The problem of weighting items 

13. The problem of scoring 

14. Differences between groups—sex, age, race, grade, environment 
etc. 


W. L. Uhl, “Some Neglected Aspects of Educational Meas- 
urement,” Journal of Educational Research, XXVII (De- 


cember, 1933), 241-246. 


Problem solving 
Social competence 
Creative ability 
Esthetic experiencing 


Peps 
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G. B. Watson, “Needed Investigations in the Psychology of 
Character,” Religious Education, XXIII (January, 1928), 
66-72. 


1. Elimination of useless educational research 

2. Investigation of methods of research 

3. What are the values and limitations of field and laboratory 
data? 

4. Shall research consist primarily of intensive work in certain 
fields or of comprehensive forms in broad fields? 

5. What are the defects and values of the psychoanalytic method 
using free reverie, formal interview, and questionnaire? 

6. How trustworthy are personal, subjective data? 

7. Use of data 

8. How shall differences between methods, experimental factors, 
and the like be stated? 

9. Repetition of important experiments 

to. What childhood factors most likely lead to shyness? 

1r, Consequences of sparing the rod, of stern fathers, fondling by 
mothers, good marks in school, of membership in a despised minority 
group 

12, General factor of character comparable with Spearman’s factor 
of general intelligence [Factor analysis, including references, was dis- 
cussed briefly in Chapter X.] 

13. Relation of reactions to musical forms to innate ability and 
training 

14. Is martial music organically or conditionally stirring? 

15. Analytical comparison of the explanations of anger offered by 
John B. Watson and by McDougall 

16. The causal factors behind concomitant learning 

17. Causes of prejudice 

18. Factors leading to unusually brave, cruel, self-indulgent, social- 
ized, or sensitive national groups 

19. Analysis of factors associated with self-denying groups and 
with groups given to immediate gratification of impulses 

20. Methods of training against playing in the street 

21. Shall scolding, spanking, explanation, visits to hospitals, elec- 
tric shocks, honor systems, mild accidents, or a combination of such 
be recommended? 

22, Immunization against propaganda 

23. Validation of techniques in group thinking 

24. Relationship between type of task and size of group 

25. Conditions prerequisite for group intelligence 

26. Reintegration of the forces of divisions of the community and 
congregation 
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“The Outlook in the Measurement of Interests,” Personnel 
Journal, IX (August, 1930), 176-183. 


ey 
S 


Fifty-one problems listed under: 


1. The development of interests and their foundations in train- 
ing and inheritance 
Interests and achievement 


3. Improving the interest inventory and its scoring 
4. Developing norms for the interest inventory 
5. Objective methods of measurement 
6. The rating scale 
7. Interests of abnormals 
8. Social, racial, and geographic differences in interests 
9. Special guidance problems 
G. SUPERVISION 
A. S. Barr, An Introduction to the Scientific Study of 
Classroom Supervision, Chapter VIII. New York: D. 
Appleton and Co., 1931. 
1. Objectives of education 
2. Data-gathering devices of supervision 
3. Factors conditioning the progress of school pupils 
4. Development of standards of practice 
5. Principles of learning, teaching, and supervising 
6. Translation of the principles of learning, teaching, and super- 


vision into learning, teaching, and supervising practices 

7. Improvability of teachers 

8. Relative effectiveness of various supervisory devices 

o. Evaluation of the products of supervision 

to. Relative effectiveness of different methods of organizing 
practice 

tr, Relative effectiveness of different modes of integrating theory 
and practice 

12. Psychology of skill in teaching 

13. Applicability of the laws of pupil learning to learning to teach 

14. Trait differences between teachers and other groups of profes- 
sional workers 

15. Factors influencing the attitude of teachers toward supervision 

16. Motivation of supervision 

17. Spread of training in supervision 

18. Factors conditioning the achievement of school pupils 

19. Characteristic differences in the teaching performance of good 


and poor teachers 
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20. Characteristic differences in the supervisory performance of 
good and poor supervisors 

21. Individual differences in teaching ability 

22. Errors, shortcomings, and difficulties of experienced and inex- 
perienced teachers 

23. Methods of educational diagnosis as applied to classroom 
supervision 


A. S. Barr, “Planning the Year’s Program,” Journal of 
Educational Research, XXII (September, 1930), 140- 
41. 


The objectives of education 

Selection and organization of subject-matter 
Nature of learning 

Provision for individual differences 

Learner’s attitude 

General welfare of school children 

Social, emotional, and character factors 
Measurement 


foci Coser ft A 


A. S. Barr, “The Scientific Study of Classroom Super- 
vision,” Journal of Educational Research, XXII (Oc- 
tober, 1930), 219-222. 


1. Scientific conception of the program of supervision as a whole 
2. Validation of data-gathering devices 
3. Mental processes involved in the different types of learning to 


4. Relative effectiveness of different training programs in super- 


5. Objective means of measuring the results of supervision 


A. S. Barr, “Needed Research in Classroom Supervision,” 
Peabody Journal of Education, V (January, 1928), 209-215. 


1. Specific information about the thing to be improved 

2. More information about the validity, objectivity, and re- 
liability of the items of teaching observed by supervisors 

3. More information about the probable effectiveness of different 
training programs 

4. Experimentally determined principles of teaching 

5. More information about the improvability of teachers 

6. More information about the relative effectiveness of various 
improvement (supervisory) devices 

7. More exact measures of teaching ability 
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T. H. Briggs, “Supervisory Experimentation,” School Re- 
view, XLI (December, 1933), 737-746. 


Certain of the problems listed are of value for practical supervisory 
purposes rather than to contribute new facts or generalizations. 


1, Extent to which explanations of assignments are understood by 
pupils 
2. A traditional form of assignment versus one in which the values 
of the expected work are stressed 
3. Value of topic assignments to small groups organized with a 
pupil chairman to prepare extended reports 
4. Effect of a written list of key questions which can be answered 
by study of the textbook and supplementary materials 
5. Value of problems and issues selected from current develop- 
ments in business, industry, government, etc. 
6. Merits of a pupil-discussion versus a textbook-recitation pro- 
cedure 
7, Evaluation of directed study 
8. Effect of hearing a radio while studying 
9. Comparison of written work prepared at home with that done 
at school 5 
to, Case study of pupils in academic difficulties and effect of spe- 
cial coaching on them 
1r. Value of work-book tests over units of instruction as directions 
for study and as self-administered measures of achievement 
12, Effectiveness of diagnostic tests followed by remedial teaching 
13. Effect of knowledge of progress on accomplishment 
14. Value of additional readings in science as compared with using 
the same amount of time for writing up experiments 
1s. Comparison of textbook-demonstration, and textbook-experi- 
mentation (by pupils) methods in science 
16. Value of voluntary laboratory work 
17. Effect of frequent short drills on work covered 
18. Comparison of drill involving the reaction of one pupil at a 
time with drill requiring responses from the entire group 
19. Comparison of rigid examinations each six weeks with daily oral 
drills in French classes 


S. A. Courtis, “Problems in the Appraisal of Supervision,” 


Educational Administration and Supervision, XV (April, 
1929), 269-278. 


1. Measurement of the amount of supervision teachers receive 
2, Measurement of the changes produced in teachers by the super- 


vision received 
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3. Measurement of the effect upon the children by the change: 
in the teachers produced by the supervision 


Bessie L. Gambrill, “A Critical Review of Researches in 
Supervision,” Educational Administration and Supervision, 
XV (April, 1929), 279-289. 


1. Studies of specific supervisory activities other than measure- 
ments 

2. Psychological factors having to do with supervisor-teacher and 
teacher-pupil relationships 

3. Home and community participation as a means of realizing 
supervisory objectives 

4. Status of supervisors and of programs for their adequate pro- 
fessional training 

5. Variations in supervisory activities to provide for individual 
differences in teachers 


Bess Goodykoontz, “The Scientific Method and Creative 
Supervision,” Educational Method, XIII (May-June, 
1934), 385-390. 


1. Improvement of teachers in service 

Curriculum construction 

Reference materials for use with the problem-solving technique 
Increased enrolment and its attendant problems 

Teaching load and teaching assignments 


Cu CS. 


M. C. S. Noble, “A Research Program for Rural School 
Supervisors,” Certain Phases of Rural School Supervision, 
pp. 12-15. Bureau of Education Bulletin No. 28, 1929. 
Washington: Bureau of Education. 


1. Relative status of educational opportunities in the county as 
compared with those of the nation and state at large 


2. Research to determine a solution of local problems 
3. Research for purposes of self-analysis and self-improvement 


Current Problems of Supervisors. Third Yearbook of the 
Department of Supervisors and Directors of Instruction 
of the National Education Association. New York: Teach- 
ers College, Columbia University, 1930. Pp. viii + 252. 


The 1682 problems fall in major groups as follows: 


A. Aims and methods of teaching 
1. Interpretation of objectives 
2. Developing social traits through pupil participation 
3. Helping pupils evaluate their own work 
4. Desirable methods 


his 


ing 
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5. Developing integrated units of subject-matter 
6. Criticism of teaching observed 

7. Observation upon request 

8, Classroom demonstrations for teachers 

9. Diagnosis of teaching difficulties 

o. Evaluation of outcomes of instruction 

11. Use of standard objective measurements 

12. Use of informal objective measurements 

13. Individual needs and difficulties 

14. Adaptation to pupil needs 


Instructional materials, supplies, and equipment 
15. Selection 

16. Preparation 

17. Interpretation 

18. Provision 

19, Suggestions concerning use 


Classroom organization 

20. The economical use of time and effort 
21. The economical use of materials 

22. The organization of pupil groups 

23. Teacher and pupil responsibility 


Professional growth in service 

24. Constructive teachers’ meetings 

25. Capitalizing best methods of teachers 

26. Codperative curriculum revision 

27. Professional growth, advancement 

28. Experimentation 

29. Improvement of limitations, professional and personal 
30. Articulation of coérdinating agencies 

31. Administrative features of supervision 


The Superintendent Surveys Supervision, Chapter VII. 
Eighth Yearbook of the Department of Superintendence. 
Washington: Department of Superintendence, 1930. 


1. Consideration of the supervisor's philosophy of the job and of 
skill 
2. Use of teacher-rating blanks to measure improvement in teach- 


Measuring changes in teaching method and skill 
Measuring changes in levels of teaching 
Elimination of errors in teaching procedures 
Elimination of teachers’ difficulties 


Kn: P 
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7. Evaluation of supervision through semi-objective rating devices 

8. Improvement in the character of the materials of instruction 

g. Objective analysis of activities in the classroom 

to. Objective analysis of characteristics of pupil behavior as indices 
of the quality of instruction 

11. Miscellaneous techniques 


H. SCHOOL ADMINISTRATION 


Carter Alexander, “Opportunities for Research in Educa- 
tional Finance,” Educational Administration and Super- 
vision, IX (April, 1923), 209-222. 


Iı. What are the available materials in a given unit, as a state or 
the city school systems of a state, and to what important uses may 
such materials be put? 

2. Just what practical and reasonably attainable changes are 
needed in records, reports, methods of reporting, time of making the 
budget, and the like, of a given unit as a state or the city systems of 
a state, to make the fiscal data thoroughly useful and why? 

3. What are the needed new techniques for securing valuable 
results from data already available? 

4. What prediction techniques are needed? 

5. How do certain educational fiscal policies really work? 

6. What can a given fiscal area afford to spend for schools, con» 
sidering its resources and its other civic needs? 

7. What are the possible and practical economies? 

8. How can adequate funds for education be raised? 


Carter Alexander, “Research in Educational Publicity,” 
Teachers College Record, XXIX (March, 1928), 479-487. 
Agents 


1. Comparative results of various publicity organizations 


Media 


2. Relation of desired responses from the public to the issuance 
of measured quantities of school publicity of determined standards 


Administration 


3. Relative values of campaign and continuous publicity 
4. Sources of financial support for school publicity 
3. Legality of tax receipts for publicity 
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6. Costs, actual and estimated, of publicity programs 
7. Proper use of pupils in publicity 


W. G. Carr, “New Angle of Attack Needed in Class Size 
Research,” Nation’s Schools, X (November, 1932), 27- 
30. 


1. Effect of large classes on pupil health 

2. Effects of more classes or longer periods on the teacher, pupils, 
and instruction 

3. Effect of large classes on mental hygiene of pupils 

4. Do teachers in small classes know their pupils better than 
teachers in larger classes, and, if so, how do they utilize this knowl- 
edge? 

s. Parents’ preferences for large or small classes and reasons for 
them 

6. Effect of large classes on failure, retardation, and acceleration 

7. Views of practising physicians, psychologists, and psychiatrists 
on the class-size problem 


M. M. Chambers, “Needed Studies in Educational Admin- 
istration,” Educational Administration and Supervision, 
XIX (April, 1933), 268-274. 


1. Statutes and judicial decisions in every state governing the em- 
ployment policies of the public-school corporation, including such 
topics as: 

(a) Religious prejudice 

(b) Partisan prejudice 

(c) Nepotism 

(d) Exclusion of married women 
(e) Teachers’ tenure acts 

(f) Teachers’ retirement systems 


Further Problems 


2. Relation of education to unworthy interests in business and 
politics 

3. Limitations upon the application of principles of business and 
industrial management to the problems of educational administration 
[Can a school be operated like an office or a factory?] 

4. Should education be one of the major coérdinate departments 
of government, along with the executive, judicial, and legislative 
branches? A 


O. L. Chapman, “Six Problems in Education,” Natiows 
Schools, XIV (December, 1934), 17-18. 
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Restatement of objectives 
Revamping of tax systems 
District reorganization 
Teacher training 
Public relations and education of the public to the program of 
the school 
6. Planning for the school plant 


T. C. Holy, “Needed Research in the Field of School Build- 
ings and Equipment,” in “The School Plant,” Review of 
Educational Research, V (October, 1935), 406-411. 


“aren 


There has been little research in the field of school buildings and 
physical equipment. The broadened curriculum, the more active meth- 
ods of learning, and emphasis upon doing and working with things 
rather than merely studying books—all have focused attention upon 
the importance of physical environment and the supply of materials 
necessary for this changed type of work. In general, the problems in 
the school-plant field fall into two major divisions: 

1. Those problems concerned with the functional planning of 
school buildings and equipment to serve better the actual instructional 
and recreational activities of the school 

(a) Space provisions necessary for various school activities 

(b) A definite basis for determining the amount of space which 
should be allowed for non-instructional parts of the build- 
ing, such as corridors, toilets, offices, storage, and similar 
areas 

2. Financial problems concerned with the efficient and economical 
construction, maintenance, and operation of the school plant—the ef- 
fect which the size of the building and type of construction have upon 
operation and maintenance costs 

(a) Is the money spent for a modern school building justified 
or po a cheaper and less elaborate building do just as 
well? 

(b) How important is the school plant to the school program, 
and in what features and to what extent do a building 
and equipment affect the efficiency of teaching? 

More detailed problems relating to buildings and equipment are: 

1. Reliable methods of determining the room requirements for a 
given program (New York state has made considerable headway in 
meeting this problem for its own program.) 

2. Size of school sites (These need to be determined on the basis 
of the recreational program.) 

Ey Flexibility in the design rl construction of school buildings 
(This problem is increasing in importance due to developments in 
curriculum and teaching methods. A school building has a life of fifty 
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to seventy-five years and unless it is of a flexible type, it either thwarts 
educational development or has an early obsolescence.) 

4. Expansibility so planned that additions can be most easily and 
economically made without destroying the unity of the building 

5. Factors which contribute to obsolescence 

6. Lighting, both natural and artificial 

7. Toilet requirements 

8. Studies of building utilization 

9. Location and plans of different types of rooms 

ro. Best building materials, when cost and utility are considered 

1. Multiple use of rooms and equipment 

12, Acoustics of classrooms, libraries, and auditorium 

13. Arrangement and types of equipment 

14. Effect of good school buildings and equipment on educational 
achievements 

1s. Methods of calculating unit costs of school buildings 

16. Effect of expenditures for capital outlay upon the school budget 
and upon the curriculum (This question is particularly pertinent in 
small communities. New buildings may furnish a stimulus to the 
support of an enlarged educational program or there may be a tend- 
ency to pay for the new building by savings made at the expense of 
teachers’ salaries, supplies, or other current expenditures.) 

17. State and city organizations responsible for the planning, erec- 
tion, maintenance, and operation of school buildings 

. 18. School plant insurance, ratio between premiums and losses 

according to types of building construction, self-insurance plans, effect 
upon insurance rates of elimination of fire hazards 

19. Continued studies of school ventilation 


G. N. Kefauver, “Proposals for a Program of Evaluation 
of Guidance,” School Review, XLII (September, 1934), 
519-526. 


1. Measurement of the need for guidance 

2. Analysis and description of practices with relation to objectives 
of guidance 

3. Comparison of practices in a school with a “standard” program 

4. Comparison of characteristics of pupils before and after ex- 
perience in guidance 

5. Measurement of characteristics of pupils after having had the 
advantage of guidance service 

6. Comparison of characteristics of pupils under various types of 
guidance service 

7. Comprehensive investigation which follows a group of pupils 
through a well-planned program of guidance and makes careful meas- 
urement at each grade level 
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G. N. Kefauver and H. C. Hand, “Needed Research in 
Guidance,” Junior-Senior High School Clearing House, 
VIII (September, 1933), 40-42. 


1. To what extent are the interests and behavior of students of 
different types affected by experiences in the various courses? 

2. Systematic analyses of the need for guidance 

3. Definition of problems of guidance 

4. Discovery and invention of new methods in guidance 

5. Effect of existing guidance procedures 


U. L. Light, “Muddling Through from Guess to Science,” 
Educational Review, LXVIII (June, 1924), 5-9. 


Kindergarten administrative problems 
Departmental teaching 

Annual or semi-annual promotion 
Class size 


ponp 


S. S. Majo, “Neglected Phases of Ability Grouping in 
High School,” American School Board Journal, LXXXI 
(July, 1930), 48. 


The author suggests that in future experiments on homogeneous 
versus heterogeneous grouping the following two phases, heretofore 
neglected, be considered: 

1. Differentiation of subject-matter 
2. Adaptation of teaching methods 


J. C. Morrison, “Problems for Investigation in State School 
Systems,” Nation’s Schools, XIII (April, 1934), 37-49. 


A summary of problems needing investigation, from the annual 
reports of state education departments. The first fifteen appeared three 
or more times, and the last twelve one or two times. 

1. Investigations to secure a more uniform accounting as a basis 
for the general improvement of specific administrative practices 

2. Studies to secure a more adequate local administrative organiza- 
ae or for the general improvement of specific administrative prac- 
ices 

3. Local studies of school-building needs as a guide for respon- 
sible local school authorities 

4. Investigations to show the inequalities in educational oppor- 
tunity or to discover a more equitable basis of educational support 

5. Diagnostic and remedial testing programs to improve the in- 
structional program in elementary schools 

6. Studies of the junior high school and of the small high school 
looking toward more effective organization 
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7. Standardization and surveys of the one-room school as the 
basis for its improvement or as a step toward its elimination 

8. Studies of the educational needs of handicapped children— 
mentally retarded, crippled, blind, with limited vision, deaf, with lim- 
ited hearing, with defective speech—as a basis for providing or im- 
proving an educational program 

9. Personnel studies—salaries, qualifications, experience, tenure, 
certification—as a basis for the improvement of the personnel service 

to, Studies of teacher supply and demand 

11. Studies looking toward a more adequate control or administra- 
tion of state textbooks 

12. Surveys of local schools or school systems as a basis for the 
improvement of the educational program 

13. Studies of transportation 

14. Guidance studies 

15. Surveys of state educational or quasi-educational institutions 

16. A critical study of the economic and social conditions under- 
lying the high-school program in Georgia 

17. An investigation of the junior-college movement throughout 
the country with special reference to the junior-college needs of Utah 

18. A survey of the agricultural problems in agricultural high- 
school communities in Oklahoma 

19. The time lost by teachers on account of illness in Florida 

20. The holding power of elementary schools in Oklahoma 

21. Overlapping in courses of study in Ohio 

22, The cost and administration of county traveling libraries in 
Florida 

23. The work of high-school parent-teacher associations in Florida 

24. Effects of educational practice upon pupil learning in Virginia 

25. A study of the value of supervision in the rural schools of 
Missouri : 

26. A study of the ways in which school art could serve industry 
in Massachusetts 

27. “A survey of health conditions of 2,000 crippled children in 
Wisconsin 


P. R. Mort, “A Forecast of Future Research in Financing 
of Education,” in “Finance and Business Administration,” 
Review of Educational Research, V (April, 1935), 165- 
170. 


1. Efficient and economical operation of schools, involving studies 
of salary schedules, class size, unit costs for categories of service, 
efficient local tax districts, budgeting, and auditing 

2. Local taxing power versus centralization, involving studies of 
local initiative and dangerous by-products of necessary centralization 
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3. Productiveness of education, involving studies of returns for 
money spent, economic value of education, and taxes based on benefit 
received 


P. R. Mort, “Needed Research in the Field of State Aid,” 
Teachers College Record, XXVII (April, 1926), 707-712. 


1. Investigation of the demands on the state fiscal program of the 
principle that the state shall encourage progress of education in the 
localities 

2. Interpretation of a satisfactory minimum educational offering 
in terms of the unit cost it involves 

3. Development of an index for measuring the burden involved 
in transporting pupils 

4. Investigation of the effect which the cost of living should have 
on a true measure of educational need 

5. Investigation of the part to be played in an equalization pro- 
gram by capital outlay 


M. C. S. Noble, “New Problems in Public School Finance,” 
American School Board Journal, LXXXIII (November, 
1931), 32, 80. 


1. Why is there a tendency from local to increased state support? 

2, Shall schoolmen advocate as the source for educational expend- 
itures certain taxes or the General Fund? 

3. Recommendations on state sources of revenue 

4. Characteristics of present taxation system 

5. Establishing standards for selecting, collecting, and distributing 
revenue 


6. Characteristics of a scientific taxation system 


J. K. Norton, “Ph. D, and Ed. D. Adventures in School 


Administration,” Teachers College Record, XXXVI (De- 
cember, 1934), 207-212, 


ata criteria for selecting problems and suggests areas for re- 
search, 


1. The organization and articulation of the administrative units 
of the American school system 
2. Relationship of laymen and lay organizations to professional 


educators in determining the content and activities of the school 
curriculum 


3. The relation of education to other ar 
tionally and structurally 
4. Determination of state funds for local education 


5. Nature, causes, effects, and promoters of demands and pressures 
on the public schools, and methods of dealing with them 


eas of government, func- 


Pany 


Dnpon n 


RobSp 


engin 
5. Personnel problems such as employment, training in service, 

rating, promotion, compensation, and tenure 
6. 
7: 
8. 
9. 
I0. 
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W. D. Parkinson, “Problems for Research,” Journal of 
Education, CX (November 4, 11, 18, 25, 1929), 381, 
383; 408-409; 438, 441; 460-462. 


Limits of compulsory attendance 

Relation between schooling and crime 
Financial support of public education 
Effect of women teachers on boys and girls 


Roscoe Pulliam, “Wanted: Some Studies of School Buying,” 
American School Board Journal, LXXVIL (November, 
1928), 38. 


Standard necessities in supplies 

Standard janitors’ supplies 

Standards of quantity 

Cost of production 

Standard list of supplies 

Lists of equipment 

H. P. Smith, “Research in the Business Management of 
Public Schools—Progress and Problems,” American 
School Board Journal, LXXVI (March and April, 1928), 
53-543 45) 129-130. 


Criteria for business practices in administration 

Adequate support under fiscal independence 

Purchasing by the municipality 

Relative efficiency of contractual and salaried janitorial and 
eering service, and their effects on plant-maintenance cost 


Appraisal of school property for insurance purposes 
Rate of depreciation on school property 
Self-insurance versus commercial insurance 
Allocations of insurance among different agencies 
A state-wide insurance fund for school buildings 


F. T. Spaulding and Others, “A List of Needed Investiga- 
tions of School Organization,” The Reorganization of 
Secondary Education, pp. 421-423. Office of Education 
Bulletin No. 17, 1932, National Survey of Secondary 
Education Monograph No. 5. Washington: Office of Edu- 


cation, 1933- 


1. The validation of a system of rating the organization of an 
individual secondary school; in particular, the determination of rela- 
tionships between: 


(a) Practice in detailed aspects of organization and pupil 
achievement 
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(b) Practice in general aspects of organization and pupil 
achievement 

(c) Practice in general aspects of organization and the judg- 
ment of competent specialists as to effectiveness of organi- 
zation 

2. Analysis of the extent to which effectiveness in one major 
phase of organization is dependent upon special effectiveness in one 
or more other phases of organization 

3. More exact analysis of the typical organization of conventionally 
organized schools, as a means of gaging the effects of reorganization 

4. Detailed analysis of the organization of outstanding private 
schools, as contrasted with the organization of public schools 

5. Study of the organization of private secondary schools used 
as agencies of public instruction, as contrasted with the organization 
of comparable public schools 

6. Analysis of various special types of organization not sufficiently 
represented in the present study to allow detailed investigation; in par- 
ticular, analysis of: 

(a) The four-year junior high-school organization within unit 
enrolment groups 

(b) The undivided five-year high-school organization within 
unit enrolment groups 

(c) Secondary-school organization forming parts of eleven- 
grade public-school systems 

(d) Secondary-school organizations which include junior-college 
units 

7. Detailed investigation of the factors which cause individual 
secondary schools either notably to exceed or notably to fall below 
the norms for organization of schools of their type and size (Study 
of this problem may properly be a phase of the validation of norms. 
On the assumption that certain norms have been accepted as valid. 
it is here suggested as a problem in the determination of causal rela- 
tionships.) 

8. Detailed study of methods by which schools necessarily han- 
dicapped either in size or in type of organization may improve their 
organization 

9. Further evaluations of various types of organization in schools 
falling in enrolment groups not adequately represented in the present 
study; in particular, study of schools separately classified: 

(a) In subdivisions of average grade enrolments below 30 
(b) In subdivisions of average grade enrolments above 333 
to. More exact analysis of the differences in organization between 
large and small schools by a segregation of schools in type groups 
and a determination of the differences between large and small schools 
common to all the type groups 
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tr. Investigation of the maximum effective size of school under 
various types of organization 

12. Study of the relationship between effectiveness of organization 
and per-pupil outlay for items immediately associated with organiza- 
tion 

13. Recurrent studies of school organization, according to methods 
similar to those employed in the present investigation, to provide: 
[Another National Survey would now be in order.] 

(a) Facts concerning the status of school organization on a 
nation-wide basis, in the light of which progress in or- 
ganization may be determined periodically 

(b) Periodic data on the relationships of the various elements 
in the national program one to another 

14. Study of the relative holding power of schools of various sizes 
and types over a period of years 

15. Investigations of the relationship between differentiation in 
salary schedules among separate units of the school system, and: 

(a) Articulation between the units 

(b) Pupil achievement in the separate units 

16. Study of the relationship between the sources of support for 
extracurriculum activities and the scope of the extracurriculum 

17. Study of the relationship between the effectiveness of homo- 
geneous groups, and the use of: 

(a) Semiannual as contrasted with annual promotions 

(b) Promotion by subject as contrasted with promotion by 
grade 

18. Investigation of the effects of extreme departmentalization in 
the junior high-school grades 

19. Comparison of both the administrative and the educational 
effects of various types of curriculum organization 


Ruth Strang, “Research Issues in Student Personnel Work,” 
Personnel Journal, XIII (August, 1934), 101-105. 


Emphasis on the value of an intensive developmental study of many 
aspects of a considerable number of individuals during the process 
of learning 

1, A cursory survey of a mass of students versus a careful analyt- 
ical study of individuals 

>, A cross section, or snapshot picture, versus a developmental 
study of individuals over a long period of time 

3. A study of end results alone versus observation of the process 
by which learning takes place 

4. Observation of an isolated fragment of the personality versus 
a consideration of the individual as a whole in his environmental 
setting 
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George D. Strayer, “The Scientific Approach to Problems 
of Educational Administration,” School and Society, 
XXIV (December 4 1926), 685-695. 


A brief summary of the development of scientific method in ad- 
ministration and a list of needed studies: 

x. Governmental control, and provisions for interpreting the de- 
sires of the community in relation to education, involving the organi- 
zation of public education in relation to national, state, and local 
government 

2. Financing of the educational program 

3. Registration of those who are subject to compulsory education, 
and the securing of their regular attendance in school 

4. Organization of schools and classes with reference to the dif- 
ferences among pupils in intelligence, physical condition, and voca- 
tional outlook 

5. Development of curricula and courses of study which take ac- 
count of individual differences and of the social objectives which the 
school is organized to secure 

6. Training, assignment, supervision, remuneration, and tenure of 
the staff employed in the schools 

7. Codperation of the schools with other social agencies 

8. Provision of buildings and equipment suitable for the most 
efficient development of the program of education to be conducted in 
them 

9. Conduct of business affairs 

to. Reporting to the public with respect to the service rendered 


David E. Weglein, “Educational Problems,” Baltimore Bul- 
letin of Education, VII (September-October, 1928), 3-5. 


1. Size of elementary-school buildings 


2. Size of junior high-school and senior high-school buildings 
3. Size of classes 

4. Length of school day and of school year 

5. Curriculum 

6. Supervision 

a 


Methods of teaching 


“Investigating the ‘Unknowns’ of School Finance,” School 
Review, XL (February, 1932), 91-92. 


A list of twenty problems expected to be studied in the National 
Survey of School Finance. 


The cost of public education 
1. Why has expenditure for education increased so markedly? 
2. Where will present tendencies lead? 
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3. How much public expenditure is really needed? 
4. What can we afford to spend for education? 


Returns for money spent 

s. Why do expenditures vary so widely from place to place? 

6. What advantages are obtained by communities spending excep- 
tionally large amounts for education? 

7. What disadvantages are suffered by communities spending ex- 
ceptionally small amounts for education? 


The tax burden for public education 

8. Under present financing systems, how is the tax burden for 
education distributed? 

9. What changes in taxation and in state and federal aid would 
bring about a more defensible distribution of burden? 


The elimination of backward areas in American education 


10. Why do they exist? 
11. What will it cost to eliminate them? 


Efficient expenditure of educational funds 

12, How can we secure greater value for what we spend? 

13. How can we effect economies? 

14. To what degree, if any, are we wasting money through the 
over-education of some boys and girls? S 

15. What is the extent of waste suffered through failure to give 
some individuals sufficient education to develop their real potentialities? 
[Bright children usually have not worked up to capacity. ] 


Public education during business depression 
16. How should education be dealt with during business depres- 


sions? 
The use of indebtedness 
17. What place is indebtedness now taking in educational finance? 
18. What place should it take? 
Public enlightenment on educational finance 
19. How can the public be continually informed on the pertinent 
questions of educational finance? 
The economic status of the teacher N 
20. What are the conditions that should determine salaries of 
teachers? 


Research Problems in School Finance. National Survey of 
School Finance. Washington, D. C.: American Council 
on Education, 1933. Pp. viii + 164. 
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t. School finance as a social factor 
2, Financial implications of the principles underlying American 
education 
3. Improvement of expenditure measurement in school finance 
4. Management of school income 
5. Securing adequate returns for school expenditures 
6. Improvement of budgeting, accounting, auditing, and reporting 
procedures 
7. Importance of private support in education 
8. Responsibilities of the state in the support of public education 
9. Responsibilities of the federal government in the support of 
public education 
10. Development of efficient local tax districts 
tr. Controls and limitations of school taxation and expenditures 
12. Stabilization of school support 


I. RURAL EDUCATION 


William G. Carr, “Some Crucial Problems in Rural Edu- 
cation,” Journal of the National Education Association, 
XIX (December, 1930), 282-284. 


Better teachers 

Longer terms and better attendance 

Greater opportunities for secondary education 

A vitalized curriculum 

Well-equipped and modern buildings 

Better understanding between rural citizens and their schools 
Adequate units of administration and supervision 

More equitable financing 


pa Gycrt feo SEF 


D. H. Kulp, “Problems of Rural Education Demanding 
Sociological Research,” Teachers College Record, XXXI 
January, 1930), 332-338. 


Policy Making and Administration 


ibys Determination of societal worths of rural groups and dominant 
institutions and agencies 


2, Population distribution, migration, trends, and tendencies in 


group memberships 


3- Division of labor between schools and non-school agencies in 
education 


4. Analysis of cultural and community areas as a basis for reor- 
ganization of curriculum units and consolidation 


5. Sociological aspects of school support 
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Teaching 


Child personalities 
Social worlds of rural children 
Effects of rural conditions on child personality 

9. Relation of maladjustments to types of social experience 

to. Qualitative tests of uses children make of school learnings in 
non-school situations, the value to children’s personalities of such 
uses for social adjustment 


wrt a 


Extracurricular Activities 


11. Study of spontaneous groupings of pupils and the correlations of 
such groupings with size of schools, geographical and social distance, 
and attitudes of teachers and parents 

12, Pupils’ out-of-school leisure time 

13. Comparison of spontaneous school groups with non-school 
leisure groups of the community 

14. Correlation of type learnings with type voluntary participations 
as a basis for programming extracurricular activities 

15. Wishes of pupils satisfied by membership in spontaneous school 
and non-school groups i 

16. New ranking methods for evaluating the worth of such partic- 
ipation, as a substitute for the practical point system 

17. Correlation of type groups and activities with levels of develop- 
ment of pupils as personalities 


School-Community Relations 


18. Community use of rural schools 

19. Shortages of facilities for group activities 

20. Analysis of types of leadership in communities 

21. Relation between teachers, school activities, and teacher leader- 


ship in various types of communities 


Teacher Training 


22. What knowledge content in rural sociology should be devised 
for teacher-training curricula? 

23. How. do field conditions of professional service determine selec- 
tion of teacher personalities? 

24. What activities do teachers carry on, that imply what socio- 
logical techniques are needed to equip them with tools for direct 
analysis of the societal aspects of their teaching situations? [There is 


real need for more sociological training on the part of teachers. | 
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Dwight Sanderson, “Status of and Prospects for Research 
in Rural Life Under the New Deal,” American Journal of 
Sociology, XLI (September, 1935), 180-103. 


I. Service research: research on specific problems, local, state, 
or national, as a basis for immediate programs of action 

2. Fact-finding and interpretation: surveys, studies of documen- 
tary materials, monographs on particular groups, etc., in an effort to 
describe the social situation and by induction to arrive at certain 
generalizations which may find specific application when properly inter- 
preted 

3. Social dynamics: a better understanding of what motivates the 
life of rural groups and the social psychology of their behavior 

4. Experimental research: careful description of the inception 
processes, and results of such social experiments as subsistence home- 
stead communities; defining objectives by study of various types of 
groups 

5. Social evaluation: demonstration of the values ‘of existing or 
proposed institutions, mores, societal forms, organizations, and proc- 
esses 


J. GENERAL PROBLEMS 


W. W. Charters, Research Problems in Radio Education. 
New York: National Advisory Council on Radio in Edu- 
cation, 1931. Pp. 17. 


I. The objectives of radio education 
1. Setting up the criteria for effective radio education 
2. Determining the relation of recreation to education in 
radio education programs 
3. Studying the limitations of radio education 
4. Studying the problems of uncensored channels for educa- 
tion by radio 
5. Determining the areas in which radio education will oper- 
ate most effectively 
JI. Administration of the station 
I. Setting up criteria for judging good administration 
Surveying stations to evaluate their efficiency 
Studying the publicity methods to secure effectiveness 
Investigating educational broadcasting finance 
Studying the internal organization of the station—staff, 
duties, control 
6. Studying the laws and regulations governing broadcasting 
7. Investigating the relations of commercial and educational 
stations 


8. Studying the relations of college stations to each other 


SS 
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Q. 


IO. 
II. 


Studying the limitations, contributions and inter-relations 
of national chains and local stations 

Securing information from audiences 

Maintaining close contact with audiences to find better 
methods 


III. Curriculum materials 


I. 


2. 
3. 


Determining criteria for effective materials of instruction 
Surveying materials to determine their effectiveness 
Diagnosing the weaknesses of materials and providing 
remedies 

Determining the tastes, needs, and wishes of audiences 
Selecting and preparing materials to be broadcast 
Selecting written materials, diagrams, pictures, and the 
like to be used in connection with broadcasts 

Preparing programs for special occasions 

Selecting extracurricular materials to be presented in 
school, before and after school, and during vacations 
Enlisting the coöperation of educators in preparing mate- 
rials 

Fitting radio lessons into school programs 

Determining grade placement of materials and grade dis- 
tribution of programs 

broadcaster and his methods of presentation 

Setting up criteria for the selection of broadcasters 
Judging the efficiency of broadcasters 

Diagnosing the weaknesses of a broadcaster and providing 
training 

Eliminating inefficient broadcasters 

Setting up criteria for good presentations 

Judging the effectiveness of presentations 

Diagnosing the weakness of presentations and providing 
remedies 

Providing broadcasters at the scheduled time 

Securing proper teaching conditions in the studio 
Making vocabulary studies 

Studying coöperation between the broadcaster and the 
classroom teacher 

classroom teacher and his methods of teaching 

Setting up criteria for the use of radio material by the 
teacher 

Surveying teachers’ methods to determine their effective- 
ness 

Diagnosing teachers’ weaknesses and providing remedies 
Securing the codperation of teachers 

Preparing classes in advance 
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. Supplementing radio instruction 
7. Studying the problems of class management introduced 
by the radio 
8. Investigating the problems of the ungraded school 
VI. The pupil and his learning 
1. Setting up criteria for good learning À 
2. Surveying classroom activities to evaluate the learning of 
the pupils 
Providing for pupil activity and participation 
Adjusting learning to the radio presentation 
Setting up proper conditions for learning 
Determining the radio interests of children 
Determining the influence of radio upon children’s reading 
[for study, recreation, and other purposes] 
VII. Adult audiences and their learning 
1. Setting up criteria for learning by adults 
2. Investigating the tastes of adults 
3. Organizing groups interested in education by radio 
4. Supplementing radio lessons 
5. Investigating the possibilities of group discussions as a 
supplement to radio lessons 
VIII. Equipment (Receiving in particular) 
1. Setting up criteria for suitable equipment 
2. Measuring the effectiveness of equipment 
3. Securing equipment (in school particularly) 
4. Maintaining equipment in good condition (for schools 
particularly) f 
5. Building equipment (by children) 
6. Studying proper physical conditions for reception 
IX. Measuring results 
1. Setting up controlled experiments in schools by 
(a) enlisting the assistance of teachers 
(b) selecting typical schools 
(c) establishing exact measure of learning 
(d) securing adequate financial support 
2. Making adequate surveys 
3. Securing audience reactions by radio, correspondence, 
questionnaire, and interview 
X. Psychology 
1. Applying the laws of learning to radio education 
2. Studying the psychology and physiological psychology of 
hearing in relation to learning over the radio to determine 
the span of attention, the proper Jength of lessons, the 
optimum time of day for lessons, the retention of material 
learned over the radio and the like 
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3. Investigating the relative effectiveness of hearing and sight 
and a combination of the two as channels for learning, to 
throw light upon the relative values of radio, silent movies, 
talkies and television for learning 

XI. History of radio education 

1. Investigating the history of the significant phases of radio 

education 
XII. Comparative radio education 

1. Investigating the objectives, policies and practices of for- 
eign countries in connection with the significant problems 
of American broadcasting 


L. D. Coffman, “The Administration of Research During 
the Depression,” Journal of Higher Education, V (Jan- 
uary, 1934), 1-6. 


Already investigations are being initiated into the effect of the 
depression on the birth rate, malnutrition among children, family rela- 
tionships, and special care of unfortunates of every kind. Studies are. 
likewise being conducted on the effect of the depression upon pre- 
school children, upon elementary-school children, secondary-school 
children, and students of college age. In addition, studies are being 
made of the effect of the depression upon school financing and its 
effect in turn upon the length of the school year, school equipment, 
and the decline of the professional status of the teacher. Medical 
schools are concerning themselves with new problems relating to 
health, growth, and disease arising out of the depression. 

Iı. Taxes and taxation 
2. Use of land reverting to the state because of delinquent taxes 
3. Economic restoration of the region in which the university is 
located 
4. Study of the employment situation 
V. E. Dickson, “The Place of Scientific Research in the 
New Education,” Educational Method, XIII (April, 
1934), 337-348. 


Deals with seven fields needing research and with means for carry- 
ing on research by four general procedures (A.) measurement, (B.) 
statistical analysis, (C.) case study, and (D.) experimental control: 

1. Mental and physical differences: measure by every means that 
will help to understand, on the one hand, the nature and extent of 
variable qualities, and, on the other hand, the nature and extent of the 
changes which take place during growth and education of the child 

2. Differences in achievement: resulting from innate capacity and 
environmental influences 

3. Problems of behavior and social adjustment: to bring about 


an equilibrium in social behavior 
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4. Differences in vocational objectives: because the new education 
stresses self-realization for each individual, there must be a research 
program in the field of “education for vocational efficiency” 

s. Problems of the curriculum: including all those educational 
activities in which the pupil is engaged 

6. Problems of teaching methods: the place of laboratory experi- 
mentation, freedom of the child, pupil initiative, drill, memorization, 
and the socialized recitation 

7. Problems of ‘administration: for example, proper size of class 
and the grouping of children 


W. C. Eells, “What Needs to Be Done Now in American 
Education?” Nation’s Schools, IX (January, 1932), 
21-26, 


The following are steps in educational progress suggested by the 
faculty of the school of education of Stanford University: 
1. The provision of a more nearly adequate and suitable means 
of financing our entire educational program 
2. The adoption of a much larger and more efficient unit of edu- 
cational organization, with professional administration and supervision 
of the schools therein 
3. The selection and training of outstanding school administrators 
[so as to rank with the leading citizens of the community] 
4. The extension and democratization of public junior colleges 
5. The adoption of the simplified calendar of thirteen equal 
months 
6. Securing adequate salaries for teachers 
7. Reform in selection, training, and certification of teachers to 
ensure proper preparation and to control the supply 
8. Revamping the whole program of teacher training in terms of 
skills to be attained rather than credits to be earned 
9. The classification of fundamental principles and functions of 
the high school, junior college, and university 
10. The revision of the curriculum to eliminate outgrown material 
and development of a unified curriculum from the kindergarten 
through the junior college 
11. Application to the college curriculum of procedures now being 
tried in elementary and secondary schools 
12. The determination of the relation of school to life, with the 
resulting effect on vocational guidance à 
13. A fuller development of the field of adult education 
14. An increased emphasis on character education 
15. The development of systems of instruction based upon objec- 
tive studies, with emphasis upon the activity principle, teaching prac- 
tice, and treatment of children pedagogically 
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16. The investigation of the principles of complex learning of the 
school type, rather than of the laboratory type 

17. The improvement of college and university teaching methods 

18, The extension of library method, personnel, and equipment 

19. The adaptation of modern inventions, such as the radio, the 
movies, the talkies, and the airplane to educational uses 

20. The development by state teachers’ associations of a forward 
looking and constructive policy dealing with fundamentals, other than 
salary increases, retiring allowances, and life tenure 


H. G. Good, “The Possibilities of Historical Research,” 
Journal of Educational Research, XXIX (October, 1935), 
148-153. 


1. A study of the transition from private to public education, 
including the American academy 

2. A study of the development of concrete and objective teaching 

3. A study of developing curricula, facilities, and methods in the 
teaching of science, the practical arts, the commercial field, and other 
subjects 

4. A history of child care and education, and the whole attitude 
toward children in the United States and the world 

5. A history of education in Nazi Germany in contrast to Republi- 
can Germany 

6. A history of educational marking and measurement 


Bess Goodykoontz, “Opportunities for the Classroom 
Teacher as a Research Worker,” School Life, XV (May, 
1930), 161-163. 


Study of methods of teaching 
Analysis of steps of learning 
Which is better—long or short daily assignments? 
Shall primary teachers use print or script? 

. Shall English teachers have a recommended or a required read, 
ing list? 

6. Does an open-book discussion in science yield better results 
than an individual-study period? 

7. Shall pupils in algebra be permitted to progress individually or 
by groups? 

8. Study of curriculum materials 

g. Case studies of pupils’ behavior 


H. Y. McClusky, “The Experimental Use of Compara- 
tive Cultures,” University of Michigan School of Educa- 
tion Bulletin, V (April, 1934), 99-101. 


1. Comparison of educational practices in the British Empire and 
America suggests many problems of instruction in grammar, com- 
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position, and literature, and in the geographical, historical, and social 
studies. 

2. Are British children more vague in their notions of the North 
American continent than American children are of the British Empire? 

3. How does instruction in British schools concerning the Ameri- 
can Revolution compare with the instruction in American schools on 
the same topic? 

4. Why is the per-pupil cost of education higher in the United 
States than it is in Great Britain? 

5. How do the careers of youth in Great Britain who do not 
continue in school beyond the age of fourteen compare with the 
careers of the corresponding youth of the same age in the United 
States who attend school until the age of sixteen? 

6. How do the attitudes of the younger generation in these two 
countries compare in certain personal, religious, economic, social, and 
international problems? 

7. What are the consequences of the fact that in the United States 
the college population is eight times as great a proportion of the total 
population as it is in England? 

8. Selection and preparation of teachers 

9. The status of the teaching profession 

to, The relationships of the teaching staff 

tr. The administrator 

12. The board of education 

13. The financial support of the schools 

14. Methods of instruction and examination 

15. The whole range of problems from the nursery school to adult 
education 


H. L. Smith, “Looking Ahead in Education,” Journal of the 
National Education Association, XXIV (September, 
1935), 177-178. 


1. The problem of what to do with the products of the schools 
as they are sent out by the hundreds of thousands year by year 

2. The school and the moral welfare of school children 

3- The need for coérdination of all agencies in education 

4. Providing a curriculum broad enough to include the social and 
vocational studies necessary to meet present-day needs 

5. Preparing our youth to meet the onslaught of radical social 
views that constantly threaten essential social stability 

6. To teach our youth it is the part of wisdom to avoid either 
one of the plans that shifts too far to one side of the central span 
of safety 

7. The successful solution of the foregoing problems will be of 
assistance in the problem of the proper training of teachers 
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G. D. Strayer, W. W. Coxe, and Others, Needed Educa- 
tional Research in New York State and Research Studies 
Recently Completed. Albany, New York: State Education 
Department, 1935. Pp. 72. 

Includes 318 studies under the following headings: 


Historical studies 
Educational history of New York State 
Educational finance 
Curriculum development 
Local educational history 
History of educational institutions 
Special methods of instruction and organization 
Educational psychology and child study 
Personality ratings 
Sex differences in behavior 
Maladjustment problems 
Group differences in various traits 
Problems of learning and thinking 
Tests and testing 
Test construction 
Regents examinations 
Prognostic measures of college success 
Critical evaluation of tests 
Special subjects of curriculum 
Art, music, and dramatics 
Commercial studies 
English, spelling, and writing 
Foreign language 
Mathematics 
Science 
Social studies 
Preschool education 
Secondary education 
Administration 
Curricula 
Teacher education 
Demand for teacher education 
Curricula 
Practice teaching 
Personnel 
Administrative 
Professional status of teachers 
Educational and social status 
Rating and characteristics of teachers 


Tenure 
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Certification 
Teacher organizations 
Miscellaneous 
Higher education 
Selection of students and predictions of success—admission of 
students 
Personnel problems 
College marks 
School administration 
Centralization of control 
Secondary-school organization 
School buildings 
School finance 
Supervision 
Transportation 
School management 
Attendance and child accounting 
Failure, retardation, and elimination 
Classification, grading, and promotion 
Curriculum studies 
Textbooks 
Health and physical education 
Social aspects of education 
Relation of social trends to education 
Parent education 
Moral and religious education 
Manual and vocational education 
Professional education 
Educational and vocational guidance 
Exceptional children 
Delinquency 
Physically handicapped children 
Mentally retarded and gifted children 
Libraries and reading 
Reading interests 
Miscellaneous 


New York State Education Department, Needed Educa- 
tional Research in New York State. Albany: University 
of the State of New York Press, 1933. Pp. 16. 
A list of sixty-three problems under the following headings: 
1. Elementary education: the curriculum, administration, and 
special classes 
2. Secondary education: organization and administration 
3. Guidance: pupil problems, special school services, health edu- 
cation, and vocational education 
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4, Rural education 

5. Higher education 

6. Teacher training and status: normal schools, teachers colleges, 
and status of teachers 
Finance 
General administration: buildings, grounds, and supervision 
Social aspects of education 
Extension education 


S29 ex 


H 


M. M. Willey, “Recent Social Changes and Educational 
Research,” Journal of Educational Research, XXVII 
(January, 1934), 321-322. 

1. Significance of population trends for education 
2. Part played by mass communication in education 


3. Taxation problems 
Place of the school in producing social values and social 


4. 
thinking 


APPENDIX II 


PROFESSIONAL PUBLICATIONS AND ORGANIZATIONS 
IN SUBJECT-MATTER FIELDS 1 


a. Art 


American Magazine of Art. American Federation of Arts, Barr Building, 
Washington, D. C. Monthly. 

School-Arts Magazine. The Davis Press (Inc.), 44 Portland Street, Worces- 
ter, Mass. 10 numbers. 

American Federation of Arts, American Art Annual. 

Association of Art Museum Directors. 

College Art Association of America, Art Bulletin (quarterly). 

Eastern Arts Association, Proceedings, Bulletin. 

Federated Council on Art Education, Reports. 

International Art Congress. 

Western Arts Association, Bulletin (6 numbers). 


b. Business and Commercial Education 


American Penman, A. N. Palmer Co., 55 Fifth Avenue, New York City. 
to numbers. 

Balance Sheet. South-Western Publishing Co., 201 W. Fourth Street, Cin- 
cinnati, Ohio. 9 numbers. 

Educator (formerly Business Educator). 612 North Park Street, Columbus, 
Ohio. 10 numbers. 

Journal of Business. University of Iowa, Iowa City, Iowa. 6 numbers. 

Journal of Business Education. Edited by Paul S. Lomax of New York 
University, New York City. (Formed by a combination of the Journal 
Be Commercial Education and the Business School Journal.) 10 num- 
ers, 

National Business Education Quarterly, Department of Business Education 
of the N. E. A., Washington, D. C. 

American Association of Collegiate Schools of Business, Proceedings, Re- 
ports. 

American Association of University Instructors in Accounting, Yearbooks. 

Commerce Guild. 

Eastern Commercial Teachers Association, Yearbooks. 

International Society for Commercial Education, 

National Association of Accredited Commercial Schools, Accredited News. 


1 See Chapter III for a brief discussion of such publications and organizations. 

For helpful references and bibliographies classified according to subject fields, see also 
J. E. Seyfried, Principles and Mechanics of Research, Chapter VI. Albuquerque, N. M.: 
University of New Mexico, 1935. 
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National Association of Commercial Teacher Training Institutions. 
National Association of Penmanship Teachers and Supervisors. 
National Commercial Teachers Federation, Yearbooks. 

N. E. A., Department of Business Education. 

New England High School Commercial Teachers Association. 

New England Penmanship Association. 

Southern Commercial Teachers Association. 

Southwestern Private Commercial Schools Association. 


c. English and Speech 

American Speech. Columbia University Press, 2960 Broadway, New York 
City. Quarterly. 

Elementary English Review. National Council of Teachers of English, Box 
67, North End Station, Detroit, Mich. 10 numbers. 

English Journal, College and High School Editions. National Council of 
Teachers of English, University of Chicago Press, 5750 Ellis Avenue, 
Chicago, Ill. ro numbers. 

English Leaflet. New England Association of Teachers of English, Cam- 
bridge, Mass. 9 numbers. 

Philological Quarterly. University of Iowa, Iowa City, Iowa. Quarterly, 

Quarterly Journal of Speech, National ‘Association of Teachers of Speech, 
University of Michigan, Ann Arbor, Mich. Quarterly. 

American Association of Schools and Department of Journalism. 

American Association of Teachers of Journalism, Journalism Bulletin 
(quarterly). 

American Society for the Study of Disorders of Speech. 

College Conference on English in the Central Atlantic States. 

Council on Education for Journalism. 

Inland Empire Education Association, Council of English. 

National Association for American Speech. 

National Association of Teachers of Speech. 

National Council of Teachers of English, English Monographs. 

National Honorary Society for High School Journalism. 

National Society for the Study and Correction of Speech Disorders. 

New England Association of Teachers of English. 

Western Association of ‘Teachers of Speech. 


d. Foreign Languages 
Classical Journal. Torch Press, Cedar Rapids, Iowa. 9 numbers. 
Classical Weekly. 1737 Sedgwick Avenue, New York City. Weekly during 


college year. 
French Review. American Association of Teachers of French, 1309 Carroll 

Street, Brooklyn, N. Y. 6 numbers. ne 
Hispania. American ‘Association of Teachers of Spanish, Stanford University, 


Calif. 4 numbers. davis 
Latin Notes. Service Bureau for Classical Teachers, New York University, 


New York City. 8 numbers. Fs 
Modern Language Journal. National Federation of Modern Language 


Teachers, 5500 Thirty-third Street, N.W., Washington, D. C. 8 numbers. 
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Modern Language Forum. Modern Language Association of Southern Cat- 
fornia, 1537 West Forty-sixth Street, Los Angeles, Calif. Quarterly. 

American Association of Teachers of French. 

American Association of Teachers of German. 

American Association of Teachers of Italian, Italica. 

American Association of Teachers of Spanish. 

American Classical League. 

Association of Modern Language Teachers of the Central West and South. 

Association of Modern Language Teachers of the Middle States and Mary- 
land. 

Classical Association of New England, Annual Bulletin. 

Classical Association of the Atlantic States. 

Classical Association of the Middle West and South. 

Classical Association of the Pacific States. 

Modern Language Association of America, Publications of the Modern 
Language Association. Quarterly. 

National Federation of Modern Language Teachers. 

New England Modern Language Association, Annual Bulletin. 

Société des Professeurs Français en Amérique, Annual Bulletin. 

Society for the Advancement of Scandinavian Study, Scandinavian Studies 
and Notes, Quarterly. 


e. Home Economics 


American Cookery, Boston Cooking School Magazine Co., 221 Columbus 
Avenue, Boston, Mass. 10 numbers, 

Journal of Home Economics. American Home Economics Association, 617 
Mills Building, Washington, D. C. Monthly. 

Parent Education. National Council of Parent Education, 60 East Forty- 
second Street, New York City. Every six weeks from October through 
May. 

Practical Home Economics. 468 Fourth Avenue, New York City. Monthly. 

American Dietetic Association. 

American Home Economics Association, 617 Mills Building, Washington, 
Due} 

N. E. A., Department of Supervisors and Teachers of Home Economics. 

New England Home Economics Association, New England News Letter. 
3 numbers. 


f. Industrial, Manual, Mechanical, and Agricultural Arts 


Industrial Arts and Vocational Education (formerly Industrial-Arts Maga- 


zine). Bruce Publishing Co., 534-44 North Milwaukee Street, Mil- 
waukee, Wis, Monthly, 


Industrial Education Magazine. Manual Arts Press, Peoria, Ill. 5 numbers. 

Journal of Engineering Education. Society for the Promotion of Engineer- 
ing Education, Prince and Lemon Streets, Lancaster, Pa. 10 numbers. 

Occupations. National Occupational Conference, 522 Fifth Avenue, New 
York City. 


Printing Education. United Typothetae of America, Fourteenth and K 
Streets, N.W., Washington, D. C. 4 numbers. 
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Vocationist. State Normal and Training School, Oswego, N. Y. Quarterly. 

American Association for the Advancement of Agricultural Teaching, An- 
nual Reports. 

American Association of Agricultural College Editors. 

American Institute of Chemical Engineers. 

American Institute of Electrical Engineers. 

American Society of Civil Engineers. 

American Society of Mechanical Engineers. 

American Management Association, Industrial Relations Association, 20 
Vesey Street, New York City. 

Association of Land-Grant Colleges and Universities, Proceedings. 

Conference of Negro Land-Grant Colleges. 

Engineering Economics Foundation, 3 Joy Street, Boston, Mass. 

Society for the Promotion of Engineering Education, Proceedings. 

Western Industrial Education Association. 

Women’s Educational and Industrial Union, 264 Boylston Street, Boston, 
Mass. 


g. Mathematics 


Mathematics Teacher, National Council of Teachers of Mathematics, 525 
West One Hundred and Twentieth Street, New York City. 8 numbers. 

School Science and Mathematics. Central Association of Science and Mathe- 
matics Teachers, Menasha, Wis. 9 numbers. 

American Mathematical Society. 

Association of Teachers of Mathematics in New England, Reports. 

Association of Teachers of Mathematics in the Middle States and Mary- 
land. 

Central Association of Science and Mathematics Teachers. 

Inland Empire Education Association, Council of Teachers of Mathematics. 

International Commission on the Teaching of Mathematics. 

Mathematical Association of America, 

National Council of Teachers of Mathematics, Year-books. 


h. Music 

Music Clubs Magazine. National Federation of Music Clubs, 320 Wait Ave- 
nue, Ithaca, N. Y. 5 numbers. 

Music Educators’ Journal (formerly Music Supervisors’ Journal). National 
Conference of Music Supervisors, 64 East Jackson Boulevard, Chicago, 
Til. 6 numbers. 

School Music. Finchley Building, Chicago, Tllinois. 5 numbers. 

Eastern Music Supervisors Conference, Eastern School Music Herald. 

Juilliard Musical Foundation, Frederick A. Juilliard, 49 East Fifty-second 
Street, New York City. 

Music Teachers National ‘Association, Proceedings. 

National Association of Schools of Music. 

National Bureau for the Advancement of Musi 
New York City. 

National Conference of Music Supervisors. 

National Federation of Music Clubs. 


ic, 45 West Forty-fiith Street, 
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National School Band Orchestra Association. 
Southern Conference for Music Education. 


i Physical and Health Education 


American Child. National Child Labor Committee, 419 Fourth Avenue, 
New York City. 10 numbers. 

American Journal of Public Health and the Nation’s Health, American 
Public Health Association, so West Fiftieth Street, New York City. 
Monthly. 

Child Health Bulletin. American Child Health Association, 50 West Fiftieth 
Street, New York City. Bimonthly. 

Hygeia. American Medical Association, 535 North Dearborn Street, Chicago, 
Tl. Monthly. 

Journal of Health and Physical Education. American Physical Education 
Association, Ann Arbor, Mich, (Formed by a merger of the American 
Physical Education Review and the Pentathlon.) 10 numbers. 

Journal of Outdoor Life. 50 West Fiftieth Street, New York City. Monthly. 

Journal of Physical Education. National Council of the Young Men’s Chris- 
tion Association, Physical Education Department, 347 Madison Ave- 
nue, New York City. 6 numbers. 

Journal of Social Hygiene. American Social Hygiene Association, 50 West 
Fiftieth Street, New York City. 9 numbers. 

Mental Hygiene. 50 West Fiftieth Street, New York City. Quarterly. 

Mind and Body. Mind and Body Publishing Co., New Ulm, Minn. 9 
numbers. 

Public Health Nursing. 50 West Fiftieth Street, New York City. Monthly. 

Recreation. National Recreation Association, 315 Fourth Avenue, New York 
City. Monthly. 

Research Quarterly of the American Physical Education Association. Ann 
Arbor, Mich. Quarterly. Devoted to fact-finding studies in the field of 
health and physical education. 

Safety Education Magazine. Educational Division, National Safety Council, 

- One Park Avenue, New York City. ro numbers. 

American Child Health Association. 450 Seventh Avenue, New York City, 
Bulletins, Proceedings. 

American Mouth Health Association. 

American Nurses Association, 450 Seventh Avenue, New York City. 

American Physical Education Association, Box 362, Ann Arbor, Mich.; 
Eastern, Middle-west, Northwest, and’ Southern Sections. 

American Public Health Association, 450 Seventh Avenue, New York City, 
Reports. 

American Student Health Association, Proceedings. 

College Physical Education Association, Proceedings, Bulletins. 

Eastern Society of the Association of Directors of Physical Education for 
Women in Colleges and Universities. 

Mid-West Association of Directors of Physical Education for Women, 
Annual Reports. 

National Amateur Athletic Federation of America, Men’s Division, 39 
Broadway, New York City, Proceedings. 
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National Amateur Athletic Federation of America, Women’s Division, 303 
West Forty-second Street, New York City, Proceedings. 

National Collegiate Athletic Association, Proceedings. 

N. E. A., Department of School Health and Physical Education. 

National Federation of State High School Athletic Associations, Proceedings. 

National Health Council, 450 Seventh Avenue, New York City. 

National Organization for Public Health Nursing, 370 Seventh Avenue, 
New York City. 

National Association of Directors of Physical Education for Women in 
Colleges and Universities. 

Society of State Directors of Health and Physical Education. 

Western Society of Directors of Physical Education for Women in Colleges 
and Universities. 

Y. M. C. A. Physical Directors Society of North America. 


j. Religious, Moral, and Character Education 


Catholic Educational Review. Catholic Education Press, 1326 Quincy Street, 
N.E., Washington, D. C. 10 numbers. 

Catholic School Interests. 178 Kenilworth Avenue, Elmhurst, Ill. Monthly. 

Catholic School Journal. 524-44 North Milwaukee Street, Milwaukee, Wis. 
Monthly. 

Character (see Religious Education). 

Character Builder. 1627 Georgia Street, Los Angeles, Calif. ro numbers. 

Character and Personality. Duke University Press, Durham, N. C. Quarterly. 

Christian Education. Council of Church Boards of Education in the United 
States, 111 Fifth Avenue, New York City. 5 numbers. 

Christian Education Magazine. Board of Education of the Methodist Epis- 
copal Church South, Nashville, Tenn. Quarterly. 

Christian Educator. Board of Education, Department of Educational Insti- 
tutions for Negroes of the Methodist Episcopal Church, 740 Rush 
Street, Chicago, Il. Quarterly. 

Christian Student. Board of Education of the Methodist Episcopal Church, 
740 Rush Street, Chicago, Til. Quarterly. 

Church School Journal. Methodist Book Concern, Cincinnati, Ohio. Monthly. 

Information Service. Department of Research and Education, Federal Coun- 
cil of the Churches of Christ in America, 105 East Twenty-second 
Street, New York City. 48 numbers. 

International Journal of Religious Education, 203 North Wabash Avenue, 
Chicago, Ill. Monthly except August. 

Religious Education. Religious Education Association, 59 East Van Buren 
St., Chicago, Ill. 2 numbers. Character. 5 numbers. 

American Catholic Philosophical Association. 

Association of Universities and Colleges of the Society of Jesus, Annual 
Reports. 

Association of Directors and Ministers of Religious Education. 

Association of Teachers of Religion. 

Character Education Institution, 3770 McKinley Street, Washington, DG 

Council of Church Boards of Education in the United States. 

Educational Association of the Methodist Episcopal Church. 


862 EDUCATIONAL RESEARCH 


Franciscan Educational Conference, Proceedings. 

International Council of Religious Education. 

National Benedictine Educational Association, Proceedings. 

National Catholic Educational Association. 

National Catholic Welfare Conference, 1312 Massachusetts Avenue, N. W., 
Washington, D. C. 

National Council on Religion in Higher Education, Yearbooks, Reports. 

National Lutheran Educational Conference, Annual Reports. 

Northern Baptist Education Society. 

Presbyterian College Union. 

Presbyterian Education Association of the South, Minutes. 

Religious Education Association, 308 North Michigan Avenue, Chicago, Ill. 

Seventh Day Baptist Education Society. 

Southern Baptist Education Association, Proceedings. 


k. Science 


Current Science. American Education Press (Inc.), 40 South Third Street, 
Columbus, Ohio. 38 numbers. 

Journal of Chemical Education. Easton, Pa. Monthly, 

Nature Magazine. American Nature Association, 1214 Sixteenth Street, 
Northwest, Washington, D. C. 

School Science and Mathematics. Central Association of Science and Mathe- 
matics Teachers, Mount Morris, Illinois. 9 numbers. 

Science, Science’ Press, Grand Central Terminal, New York City. Weekly. 

Science Education. Lime and Green Streets, Lancaster, Penna. (Formerly 
General Science Quarterly.) Quarterly. 

Scientific Monthly. Science Press, Grand Central Terminal, New York City. 
Monthly. 

American Association for the Advancement of Science, Washington, D. C. 
Proceedings. 

American Chemical Society. 

American Chemical Society, Division of Chemical Education. 

American Chemical Society, Senate of Chemical Education. 

American Institute of Chemical Engineers. 

Botanical Society of America. 

Central Association of Science and Mathematics Teachers. 

Eastern Association of Physics Teachers, Reports. 

Geological Society of America. 

National Academy of Sciences, B and Twenty-first Streets, Washington 
D. C., Proceedings, Annual Reports. 

National Association for Research in Science Teaching. 

N. E, A., Department of Science Instruction. 

New England Association of Chemistry Teachers, Reports. 


1, Social Studies 


Current Events. American Education Press (Inc.), Columbus, Ohio, 38 
numbers. 
Economic Geography. Clark University, Worcester, Mass. Quarterly. 
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Journal of Educational Sociology. New York University, New York City. 
9 numbers. 

Journal of Geography. National Council of Geography Teachers, Menasha, 
Wis. ọ numbers. 

National Geographic Magazine. National Geographic Society, Washington, 
D. C. Monthly. 

Social Forces. Williams and Wilkins Co., Baltimore, Md. Quarterly. 

Social Science, Pi Gamma Mu, National Social Science Honor Society, 1414 
East Fourth Avenue, Winfield, Kan. Quarterly. 

Social Science Abstracts, Social Science Research Council, Columbia Uni- 
versity. Monthly (discontinued in 1932). 

Social Studies (formerly Historical Outlook). McKinley Publishing Co., 
Philadelphia, Pa. 8 numbers. 

Sociology and Social Research, University of Southern California, Los 
Angeles, Calif. Bimonthly. 

American Academy of Political and Social Science, 3622-24 Locust Street, 
Philadelphia, Pa., Annals (bimonthly). 

American Association of Social Workers, 130 East Twenty-second Street, 
New York City. 

American Catholic Historical Association. 

American Economic Association. 

American Geographical Society, Broadway at One Hundred and Fifty-sixth 
Street, New York City. 

American Historical Association, 40 B Street, S.W., Washington, D. C. 

American Political Science Association. 

American Social Hygiene Association, 450 Seventh Avenue, New York City. 

American Sociological Society, Proceedings. 

Association of American Geographers. 

Association of History Teachers of the Middle States and Maryland, Pro- 
ceedings. 

Association of Schools of Professional Social Work. 

Bureau of Commercial Economics. 

Mississippi Valley Historical Association. 

National Conference of Social Work, 277 East Long Street, Columbus, 
Ohio, Proceedings, Quarterly Conference Bulletin. 

National Council of Geography Teachers. 

N. E. A., Department of Social Studies. 
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